Rater training programs aim to minimize rating errors and improve performance evaluations. The article discusses four types of rater training: Rater Error Training teaches raters to recognize common errors; Frame of Reference Training instructs raters on performance standards; Behavioral Observation Training improves observation skills; and Self-leadership Training enhances self-awareness. Three articles analyze the effects of rater accountability, rating format, and accuracy incentives on evaluation quality. A case study examines issues with Yahoo's forced ranking system. Overall, the presentation addresses how to conduct effective performance evaluations through rater training and rating design.
1. Running head: EXECURTIVE SUMMARY
1
2
EXCUTIVE SUMMARY
Executive Summary 2
Covering Week 4
Psyc601, Fall 2017
Tong Yao
California State University, San Bernardino
2. Executive Summary 2
The readings assigned last week were chapter two and
three from the book titled Performance Management. These two
chapters were complementary with class lecture, activity, and a
presentation. In class, we have talked about the six stages of
performance management in chapter two. There are two
important prerequisites came in place before the process of
performance management: knowledge organization’s mission
and goal, and knowledge of the job through strategic planning.
The strategies help the organization to identify their purpose,
and help the organizations to reach their goals.
Performance planning includes result and behaviors as well
as a development plan. Results is the expectations or the
outcomes that the organization is aiming for. Performance
standards are also important because they provide information
about acceptable and unacceptable performance, so that
employees can be evaluated on how well they have achieved
their goals. Behaviors is measured through competencies, which
determines the results.
Performance execution comes after the planning.
Employees need to commit to the established goals. Feedback
and coaching should be ongoing and continues through the
process. One way to help with feedback and coaching is to
collect and share performance data, and be prepared with
performance reviews. Employers need to observe and document
performance on a daily basis. Employers should also provide
employees with resources and opportunities, and reinforce
employees’ outstanding performance.
3. Performance review should focus on the past, present, and
future. It is necessary to know what has been done and how,
what compensation is received or denied as results, and goals in
the future. There are six steps help with conducting productive
performance reviews as follows: Identify strength and weakness
of employee’s behaviors, give feedback about these behaviors,
the importance of changing these behaviors. The Supervisor and
employee must agree on the action plan, and set up follow- up
meetings. Performance renewal and recontracting is similar with
performance planning; however, it is based from previous
phases, and performance management cycle restarts after this
process.
After the lecture on performance management process, a
class activity helped to grasp the concepts and apply them in
practice. Omega Inc. lacked all of the management process.
They must have a job description before performance planning.
It also lacked goals, feedback and standard appraisal, so it is
important to set formal meetings and set up goals. If the needs
are not met, it is important to change the goals at the stage of
renewal and recontracting.
The presentation on chapter three gives a general idea about
performance management and strategic planning. The first
article demonstrates both employees and organizations need to
have mission, vision, goals and strategies to improve
performance, and must clearly align and be congruent based on
organizations and unit level. The second article demonstrates
that the employee needs to understand their key accountabilities
or their responsibility to meet the organization’s goals.
Supervisors and employees need to agree on the areas of the
development plan and must set goals before implementation.
The third article is about how strategic performance
implementation issues affect strategic performance
management. There are three important factors: development of
information architecture, alignment with incentives, and
mission, vision, and goals. These factors determine the
effectiveness of strategic performance management in Indian
4. Oil Industry.
Chapter three explains that Strategic planning is linked
with performance management. The purpose for strategic
planning is to help organizations in pursuit of its goals. The
main point is to identify external (opportunities and threats) and
internal (strengths and weakness) trends. The internal can be
organizational structure, and processes. External are based on
economic, and technological. An opportunity plus strength is
called leverage, opportunity with weakness is called constraint,
threat plus strength is vulnerability, threat and weakness is
called problem.
Both employees and organizations need to have mission, vision,
goals and strategies to improve performance, and must clearly
align and be congruent based on organizations and unit level.
Job description, behaviors and results are also crucial. Mission
statement includes the technology that is used in productivity
and the benefits of the products and services. A good vision
must be brief, verifiable, bound by a timeline, current, focused,
understandable, inspiring, and stretch. Goals provide specific
implements to achieve mission and vision. The strategies are
growth, survival, turnaround, stability, innovation, and
leadership.
In conclusion, the key component is to understand the activities,
tasks products, services, and processes of the job. Evaluations
cannot be done without these criteria. Also, knowledge of the
job is necessary as well before the implement of performance
management system. It is important to update goals, objectives,
and accountabilities as well as feedback toward these standards.
Recontracting is an ongoing process. Poor implement can affect
the whole system, so that links between each component must
be clear. Then, moving to Strategic planning, the purpose is to
help both organizations and employees achieve goals through
effective approaches. It helps employees to adapt to
environmental change, guide them directly in daily basis, and
even to build better allocation of resources on the
organizational levels. A mission statement should include
5. organizational values and beliefs. Vision is also an important
component, which includes the future aspirations. Identifying
strategies that help achieve goals is crucial too. Have an
understanding of “what’s in it for me?” helps carry out the
understanding of the system.
Rating Issues
Arturo Covarrubias-Paniagua
Hello, I’m Arturo and I’m here to present on Rating issues.
1
Overview
Model of Rating Behavior
3 Articles
Carmbon & Steiner (2015)
Harari & Rudolph (2016)
Jelley, Goffin, Powell, & Heneman (2012)
Case Study
How to reduce rating error
Today we will briefly review the model of rating behavior
again, 3 articles covering issues in rating, a case study
regarding a PM system implemented in Yahoo, and some tips on
how to reduce error.
2
Model for Rating Behavior
6. A model – Rating behaviors are influenced by a motivation to
provide accurate ratings and the motivation to distort ratings.
3
Article 1
Incentives and Alternative Rating Approaches: Roads to Greater
Accuracy in Job Performance Assessment?
Jelley, Goffin, Powell, & Heneman (2012)
4
Background
Providing an accuracy incentive prior to observing performance
can effect raters.
Format of rating (serial vs. parallel) can affect ratings.
5
Types of Rating Accuracy & Serial vs Parallel
Elevation Accuracy (EL) – Raters level-of-rating accuracy
Differential Elevation (DE) – The differential main effects of
ratees.
Stereotype Accuracy (SA) – concerns the raters accuracy in
differentiating among items
7. Differential Accuracy (DA) – represents the differential ratee-
by-item interaction.
Serial – Encourages raters to consider relative judgments of
specific behavior (one subject at a time)
Parallel – Rate all ratees on a given behavior (all subjects at
once)
Poor EL reflects a raters tendency to evaluate ratees too high or
too low
DE indicates a raters accuracy in differentiating among ratees in
terms of their general level of performance
SA – is relevenat to group-level training needs assessment
DA – relevant for individual level identification of patterns of
strengths and weaknesses among ratees behaviors
Now, there are also two different ways to go about rating
multiple people at once. You can do serial rating, where you
basically rate one person on all of their judgements at one time,
while parallel has you rate all ratees on a certain dimension at
once.
6
Hypothesis
H1: Serial approach will be accurate in all ways (DA, SA, EL,
DE) then the parallel approach in the no-incentive condition
H2A: Parallel approach will be more accurate (EL & DE) than
the serial approach in the recall-incentive condition.
H2B: Parallel approach will be more accurate in all ways (DA,
SA, EL, DE) then the serial approach observation-incentive
condition.
8. 7
Methods
147 student participants
Viewed recording of lecturers lectures, rated 48 hours later.
Some students were told that they would receive 20.00$ if they
were accurate, either before or after
Others were not told at all of an incentive
8
Results & Implications
Serial rating approach was more accurate than the parallel
approach.
Accuracy incentive before the observation mitigated the effects
of the parallel approach.
9
Article 2
The effect of rater accountability on performance ratings: A
meta-analytic review
Harari & Rudolph (2016)
The second article I’ve chosen covers the effects of rate
accountability on performance ratings, by Harari & Rudolp
10
9. Background & Hypothesis
Rater Accountability
Accountability is a norm in most appraisal systems, but not
universal.
Sometimes ratings are not shared with ratees.
The effect of rater accountability mechanisms remains unclear
Firstly, Rater accountability is when another individual holds
raters responsible for their performance ratings. People
experience accountability when they believe that their
performance ratings have implications for their social image
with their organization. The article lists an example of
accountability as a situation where raters need to let the ratee
know the reasons and results of their rating. As such,
accountability is typically the norm in most appraisal systems. I
think a real life example would be the SOTE’s.
The article however notes that the effects that rater
accountability has on ratings is still somewhat unclear, so they
undertook a meta-analysis.
11
Methods
Initial search yielded 138 articles.
Manipulation of rater accountability
D-value indexing
Performance ratings as a dependent variable
Analyzed 35 independent samples.
When they began their literature search they originally found
138 articles that met their search criteria. However; they
narrowed these articles down by requiring them to have a
manipulation of rater accountability, enough information so that
10. the reviewers could calculate d-values, and performance ratings
needed to be the dependent variable in the study.
After they narrowed down their results they have 35 original
papers to conduct the analyses.
12
Results & Implications
Cohen’s d = .12, very small effect size
When accounting for accountability source (ratee vs. superior)
as a moderator, effect sizes changed
Ratee Cohen’s d = .32, small to medium
Supervisor Cohen’s d = -0.06.
Performance ratings were higher when Ratee’s were
accountable.
Being held accountable by a ratee results in inflated
performance ratings.
Their initial results on the effect of accountability alone on
performance ratings was small, indicating that accountability
does have a small effect on increasing the scores. However,
when they took the accountability sources as a moderator, they
found that the effect size for ratees was small to medium, while
the effect size for superiors was practically not there at all.
Overall, being held accountable by a ratee results in inflated
ratings.
13
Article 3
When Rating Format Induces Different Rating Processes
Cambon & Steiner (2015)
11. 14
Background
Rating Format
Descriptive Behavior
Evaluative Behavior
Big Five Personality
Conscientiousness and Extraversion.
“Agency”
Cronbach’s distinct forms of accuracy
DA
SA
EL
SE
The main goal is to examine different rating formats and their
interactions with the purposes of rating (administrative vs.
developmental), the induced performance rating processes and
their effects on the accuracy of the rating.
Now lets take a step back and cover some of these concepts.
Firstly the authors state that different rating formats should
serve different functions in performance appraisal and in
general psychological assessment. In terms of the authors focus
on rating scales that use either descriptive behavior, where it
focuses on the characteristics of the target. The second format is
evaluative behavior because they mainly entail looking at the
target based on an evaluation of what one can do with the
target, while saying nothing about the character of it.
This leads to their first hypothesis that a rating format based on
DBs will produce more within-rate discriminability than a mode
12. based on EB. And a rating mode based on EB’s will produce
more between rate discriminability than a mode based on traits.
Secondly, the article focuses on the dimensions of performance.
They note earlier researches focus between judgments of
communication or agency. Where communion refers to
interpersonal behaviors related to socialization and friendship,
where agency refers to behaviors related to power and personal
growth. In current terms communion matches up with the
dimensions of agreeableness and neuroticism, whereas Agency
matches up really well with the dimensions of conscientiousness
and extraversion in the big five. This leads to their next
hypothesis that ratings made on the agency dimension will
produce more between-rate discrimibailty than ratings made on
the communion dimension.
15
Methods
Two experiments
Participants rated seven targets presented via videotapes
Given descriptive knowledge, evaluative knowledge, or a mix of
the two.
Indexes of discriminability and of accuracy.
16
Results & Implications
Results indicated that EB ratings scales had higher between-rate
discriminability
DB rating scales had higher within-rate discriminability
Use EB to compare different ratees.
13. Use DB to identify strengths and weakness of a ratee
17
Case study – Yahoo’s Performance Management System
Mayer introduced a system known as QPRs (Quarterly
Performance Reviews). Through this popular performance
review technique, managers would set and communicate goals
and results to the departments, teams and individual employees.
Employees would get a score every quarter from one to five. A
one meant the employee was consistently missing their goals,
while a five meant that they were greatly exceeding their goals.
In essence, the system is similar to stack ranking (as seen at GE
& Adobe) The target distribution system or stack ranking, put
employees in five buckets. Ten per cent of high performing
employees would go into “greatly exceeds,” twenty-five in
“exceeds,” fifty percent into “achieves,” ten percent into
“occasionally misses,” and five percent into “misses.”
Seventy-five% of the entire company got into the top three
buckets, while 25 percent of every team had to go into the
bottom two. This results in an incredibly competitive work
environment, where teammates directly competed with each
other to make sure they didn't end up in the bottom 25 percent.
“I was forced to give an employee an occasionally misses, was
very uncomfortable with it. Now, I have to have a discussion
about it when I have my QPR meetings. I feel so uncomfortable
because in order to meet the bell curve, I have to tell the
employee that they missed when I truly don’t believe it to be
the case. I understand we want to weed out mishires/people not
meeting their goals, but this practice is concerning. I don’t want
to lose the person mentally. How do we justify?”
14. Now with this in mind, are some of the big problems and how
would you rectify them?
Get rid of forced choise. Implement possible
18
Summary
Serial ratings are generally better than parallel approaches, but
accuracy incentives mitigate the issue.
Being held accountable by a ratee results in inflated
performance scores
EB scales should be used to compare amongst ratees, DB scales
should be used for developmental purposes.
Serial rating approach was more accurate than the parallel
approach.
Accuracy incentive before the observation mitigated the effects
of the parallel approach.
Serial rating methods are better for developmental purposes.
19
References
Jelley, R., Goffin, R., Powell, D., & Heneman, R. (2012).
Incentives and Alternative Rating Approaches: Roads to Greater
Accuracy in Job Performance Assessment? Journal of
Personnel Psychology, 11(4), 159-168.
Cambon, L., & Steiner, D. (2015). When Rating Format Induces
Different Rating Processes: The Effects of Descriptive and
Evaluative Rating Modes on Discriminability and Accuracy.
Journal of Business and Psychology, 30(4), 795-812.
Harari, & Rudolph. (2017). The effect of rater accountability on
15. performance ratings: A meta-analytic review. Human
Resource Management Review, 27(1), 121- 133.
http://allthingsd.com/20131108/because-marissa-said-so-
yahoos-bristle-at-mayers-new-qpr-ranking-system-and-silent-
layoffs/
https://blog.impraise.com/360-feedback/is-this-the-end-of-
yahoo-and-employee-stack-ranking-performance-review
20
Rater Issues in Performance Management system
Presented By: tong yao
1
Overview
Why rater training programs?
Training programs
Rater Error Training (RET)
Frame of Reference Training (FOR)
Behavioral Observation Training (BO)
Self-leadership Training (SL)
3 articles
Case study
Conclusion
Reference
16. 2
Why Raters Training Programs?
To identify job activities
To measure performance
To minimize rating error
To train counsel and coach
To interact with employees
To prepare for performance management system.
To increase satisfaction with the system.
3
Training Programs
Rater Error Training (RET)
Frame of Reference Training (FOR)
Behavioral Observation Training (BO)
Self-leadership Training (SL)
4
Rater Error Training
To make rater aware of what rating errors they are likely to
make and to minimize them:
Intentional:
Leniency (inflation): high ratings to most or all employees.
Severity (deflation): low ratings to most or all employees.
Central tendency: use only the middle points on the rating
17. scales.
5
Rater Error Training
Unintentional:
Similar to Me: favor who are similar with him or herself.
Halo: high score on one dimension, high score on all other
dimensions.
First Impression: make an initial favorable or unfavorable
judgment.
Primacy: gives more weight toward the beginning of the review
period.
Recency: gives more weight toward the end of the review
period.
6
Rater Error Training
Contrast: compare individuals with one another.
Stereotype: oversimplified view of individuals based on group
membership.
Negativity: more weight on negative observed than on positive
and neutral information.
Spillover: scores from previous unjustly influence current
ratings.
Attribution: attributes poor performance to an employee’s
dispositional tendencies.
18. 7
Frame of Reference Training (FOR)
Goals for FOR:
Improve rater accuracy by thoroughly familiarizing raters with
the various performance dimensions to be assessed.
Provide accurate ratings of each employee on each dimensions.
Observing Performance
Evaluating performance
8
Behavioral Observation Training (BO)
Goals for BO:
Minimize unintentional rating errors
Improve rater skills by focusing on how raters:
Observe performance
Store information about performance
Recall information about performance
Use information about performance
9
Self-leadership Training (SL)
Goals of SL
Improve rater confidence in ability to manage performance
Self-direction
Self-motivation
confidence
19. Enhance mental processes
Increase self-efficacy
10
Article 1
The Influence of Collectivism and Rater Error On
Organizational Citizenship and Impression Management
Behaviors
Kim, P., & Lee, J. (2012).
11
What is the study about?
In this study the aim was to distinguish empirically between
organizational citizenship behavior (OCB) and impression
management behavior (IMB), and to examine the relative
contributions of collectivism and rater error to those behaviors.
12
Definitions
Organizational citizenship behavior(OCB): prosocial
organizational behavior (Brief & Motowidlo, 1986), civic
organizational behavior (Graham, 1991), organizational
spontaneity (George & Brief, 1992), intraorganizational
volunteerism (Peloza & Hassay, 2006), and contextual
performance (Borman & Motowidlo, 1993).
Impression management behavior(IMB) is a behavior that
20. individuals employ to protect their self-image or influence the
way they are perceived by others (Schlenker, 1980).
13
Definitions
Individualism: refers to a self-orientation, an emphasis on self-
sufficiency and control, and the pursuit of individual goals that
may, or may not, be consistent with group goals (Wagner, 1995)
Collectivism: involves subordination of personal goals to those
of a larger group and emphasizes sharing, cooperation, group
harmony, and concern for group welfare (Wagner, 1995)
14
Hypotheses
Hypothesis 1: An individual’s collectivism will be positively
associated, and individualism will be negatively associated,
with organizational citizenship behavior.
Hypothesis 2: An individual’s collectivism will be negatively
associated, and individualism will be positively associated, with
impression management behavior.
15
21. Hypotheses
Hypothesis 3: Perceived rater error will be negatively
associated with organizational citizenship behavior.
Hypothesis 4: Perceived rater error will be positively associated
with impression management behavior.
Hypothesis 5: Perceived rater error will be negatively
associated with in-role behavior.
16
Perceived Rater Error and Organizational Behaviors
Employee’s IMB can affect a rater’s judgment, individuals may
use IRB, OCB, and IMB strategically to achieve a higher
performance rating.
When a performance appraisal is biased, distorted, or
inaccurate, the chances of increasing the productivity of an
employee are largely limited.
People who perceive unfairness are likely to withhold or reduce
their contribution to the organization, and, thus, have lower
OCB
17
Results
Hypothesis 1 & 2: Found that there was a positive relationship
between collectivism and impression management behavior
(IMB).
Hypothesis 3, 4 &5: Perceived rater error tends to make
employees decrease their organizational citizenship behaviors
22. (OCBOs) and in-role behaviors (IRBs).
18
Strategies
Rater training
Redesign of rating process
Climate of fair treatment for employees (Folger & Greenberg,
1985; Werner, 2000).
19
Article 2
Morning Employees Are Perceived as Better Employees:
Employees’ Start Times Influence Supervisor Performance
Ratings.
Yam, K.C., Fehr, R., & Barnes, C.M. (2014)
20
What is the study about?
Stereotyping literature to suggest that supervisor ratings of job
performance are affected by employees’ start times.
Later start time leads supervisors to perceive employees as less
conscientious. These perceptions in turn cause supervisors to
rate employees as lower performers.
Supervisors should remain mindful of the morning bias and
ensure that their performance ratings are based on more reliable
performance metrics
23. 21
Hypotheses
Hypothesis 1: Employee start time is negatively associated with
supervisor perceptions of employee conscientiousness.
Hypothesis 2: The relationship between employee start time and
supervisor ratings of job performance is mediated by supervisor
perceptions of employee conscientiousness.
Hypothesis 3: The indirect effect of employee start time on
supervisor ratings of performance via perceived conscien-
tiousness is moderated by supervisors’ chronotypes, such that
the indirect effect will be strong when supervisors are “larks,”
but dissipate when supervisors are “owls.”
22
Results
Study 1: Hypothesis 1: participants display a morning bias and
automatically associate morningness with conscientiousness.
Study 2: Hypothesis 1, 2 & 3: employee start times are
meaningfully related to supervisors’ performance ratings.
Study 3: Hypothesis 3: When participants were larks, the
mediated model was significant.
When participants were owls, however, the mediated model was
not significant
24. 23
Strategies
flexible work practices(FWP): allow employees to meet
obligatory duties, while enabling organizations to attract and
retain talent by enhancing employees’ job satisfaction and
commitment to the organization.
It helps supervisors better understand how they rate their
employees’ performance.
Rater Error training
24
Article 3
Does Rater Personality Matter? A Meta-Analysis of Rater Big
Five- Performance Rating Relationships
Harari, M. B., Rudolph, C. W., & Laginess, A. J. (2015)
25
What is the study about?
To find out the role of rater’s personality in rating:
To examine rater personality traits consistent with the Five-
Factor Model as sources of systematic non-performance
25. variance in job performance ratings using meta-analysis
to examine the contextual as moderators of the rater
personality–performance rating relationships
26
Results
Across 21 studies and 28 independent samples, found evidence
suggesting that rater personality traits consistent with the Five-
Factor Model accounted for between 6% and 22% of the
variance in performance ratings.
rater agreeableness, extraversion, and emotional stability were
all positively related to performance ratings.
raters who are high in openness respond to the performance
rating context by considering performance episodes and drawn
about the employee’s performance.
the effect of rater personality on performance ratings should be
stronger in field studies as compared to laboratory studies
27
Strategies
Rater training may be considered strong and therefore may also
be likely to reduce rater personality –performance rating
relationships
Rater–ratee personality similarity could potentially influence
performance ratings through its effect on psychological distance
Rater error training
Similar-to-me error
Frame of reference training
26. 28
Case Study 4 (p.g 199)
Using the information in section 4, traning programs for the
acquisition of required skills
a brief explanation of the nature of your suggested rater training
program.
Its advantages?
Its requirements?
29
Conclusion
We reviewed the following points;
Issues regarding rater performance based on three articles
Importance, sources and needs of rater training programs
Types of training programs that should incorporated
How to overcome raters’ issues
Combine different training programs
30
References
Aguinis, H. (2013). Performance management. Upper Saddle
River, NJ: Pearson Prentice Hall.
27. Harari, M. B., Rudolph, C. W., & Laginess, A. J. (2015). Does
rater personality matter? A meta‐analysis of rater Big Five–
performance rating relationships. Journal Of Occupational And
Organizational Psychology, 88(2), 387-414.
doi:10.1111/joop.12086
Kim, P., & Lee, J. (2012). The influence of collectivism and
rater error on organizational citizenship and impression
management behaviors. Social Behavior And Personality, 40(4),
545-556. doi:10.2224/sbp.2012.40.4.545
Yam, K.C., Fehr, R., & Barnes, C.M. (2014). Morning
employees are perceived as better employees: Employees’ start
times influence supervisor performance ratings. Journal of
Applied Psychology, 99(6), 1288-1299
31
Psyc601 - K. Shultz, Week 6
1
1
1
1
PSYC601 – Week 6
Gathering Information and Implementation
Tong’s Presentation – Rater Issues
Gathering Information in PM (Ch 6)
Case Study 6-3
BREAK
Arturo’s Presentation – Rating Issues
Implementation of the PM Process (Ch 7)
Modified Case Studies 7-2 and 7-3
Things to come
28. 1
Psyc601 - K. Shultz, Week 6
2
Appraisal Forms:
8 Desirable Features
Simplicity
Relevancy
Descriptiveness
Adaptability
Comprehensiveness
Definitional Clarity
Communication
Time Orientation
Psyc601 - K. Shultz, Week 6
3
Determining Overall Rating
Judgmental strategy
Holistic judgments – with defensible summary
Mechanical strategy
Weighted summary based on relative importance
Psyc601 - K. Shultz, Week 6
4
29. Appraisal Period and Timing
Number of Meetings
Annual, Semi-annual, or Quarterly
Anniversary Date
Supervisor doesn’t have to fill out forms at same time, but
Can’t tie rewards to fiscal year
Fiscal Year
Rewards tied to fiscal year
Goals tied to corporate goals, but
May be burden to supervisor, depending on implementation
Psyc601 - K. Shultz, Week 6
5
Who Should Provide
Performance Information?
Employees should be involved in selecting
Which sources evaluate
Which performance dimensions
When employees are actively involved
Higher acceptance of results
Perception that system is fair
Those with direct knowledge of employee performance should
be used
Supervisors, Peers, Subordinates, Self, Customers (both internal
and external)
Psyc601 - K. Shultz, Week 6
6
30. Disagreement Across Sources
Expect disagreement
Ensure employee receives feedback by source
Assign differential weights to scores by source, depending on
importance
Ensure employees take active role in selecting which sources
will rate which dimensions
Psyc601 - K. Shultz, Week 6
7
Types of Rating Errors
Intentional errors
Rating inflation
Rating deflation
Unintentional errors
Due to complexity of task
Psyc601 - K. Shultz, Week 6
8
Expected Positive and Negative Consequences of Rating
Accuracy
Probability of Experiencing Positive and Negative
Consequences
Expected Positive and Negative Consequences of Rating
Distortion
Probability of Experiencing Positive and Negative
31. Consequences
Motivation to Provide Accurate Ratings
Motivation to Distort Ratings
Rating Behavior
A Model of Rater Motivation
Psyc601 - K. Shultz, Week 6
9
Rater Training Programs
Should Cover
Information on how the system works
Motivation – What’s in it for me?
Identifying, observing, recording and evaluating performance
How to interact with employees when they receive performance
32. information
Psyc601 - K. Shultz, Week 6
10
Case Study – 6.3
Based on what we now know about rater training programs, rate
each content area in terms of whether they are intentional error
or unintentional errors.
Psyc601 - K. Shultz, Week 6
11
Break
Psyc601 - K. Shultz, Week 6
12
Implementing a Performance Management System: Overview
Preparation
Communication Plan
Appeals Process
Training Programs
Pilot Testing
Ongoing Monitoring and Evaluation
33. Psyc601 - K. Shultz, Week 6
13
Preparation
Need to gain system buy-in through:
Communication plan regarding Performance Management
system
Including appeals process
Training programs for raters
Pilot testing system
Ongoing monitoring and evaluation
Psyc601 - K. Shultz, Week 6
14
Communication Plan Answers
What is Performance Management (PM)?
How does PM fit in our strategy?
What’s in it for me?
How does it work?
What are our roles and responsibilities?
How does PM relate to other initiatives?
Psyc601 - K. Shultz, Week 6
15
Cognitive Biases that Affect
Communications Effectiveness
Selective exposure
What you see?
Selective perception
34. What you perceive?
Selective retention
What you retain?
Psyc601 - K. Shultz, Week 6
16
To minimize effects
of cognitive biases
A. Consider employees
Involve employees in system design
Show how employee needs are met
B. Emphasize the positive
Use credible communicators
Strike first – create positive attitude
Provide facts and conclusions
C. Repeat, document, be consistent
Put it in writing
Use multiple channels of communication
Say it, and then – say it again
Psyc601 - K. Shultz, Week 6
17
Appeals Process
Promote Employee buy-in to PM system
Amicable/Non-retaliatory
Resolution of disagreements
Employees can question two types of issues
Judgmental -validity of evaluation
Administrative-whether policies and procedures were followed
35. Psyc601 - K. Shultz, Week 6
18
Appeals Process
Level 1
HR reviews facts, policies, procedures
HR reports to supervisor/employee
HR attempts to negotiate settlement
Level 2
Arbitrator (panel of peers and managers) and/or
High-level manager – final decision
Psyc601 - K. Shultz, Week 6
19
Rater Training Programs
Content Areas to include
Information
Identifying, Observing, Recording, Evaluating
How to Interact with Employees
Choices of Training Programs to implement
Rater Error Training
Frame of Reference Training
Behavioral Observation
Self-leadership Training
Psyc601 - K. Shultz, Week 6
36. 20
Content
A. Information - how the system works
Reasons for implementing the performance management system
Information
the appraisal form
system mechanics
B. Identifying, observing, recording, and evaluating
performance
How to identify and rank job activities
How to observe, record, and measure performance
How to minimize rating errors
C. How to interact with employees when they receive
performance information
How to conduct an appraisal interview
How to train, counsel, and coach
Psyc601 - K. Shultz, Week 6
21
Choices of Training Programs
Rater Error Training (RET)
Frame of Reference Training (FOR)
Behavioral Observation Training (BO)
Self-leadership Training (SL)
Psyc601 - K. Shultz, Week 6
22
Intentional Rating Errors
Leniency (inflation)
Severity (deflation)
37. Central tendency
Psyc601 - K. Shultz, Week 6
23
Unintentional Rating Errors
Similar to Me
Halo
Primacy
First Impression
Contrast
Stereotype
Negativity
Recency
Spillover
Attribution
Psyc601 - K. Shultz, Week 6
24
Frame of Reference Training (FOR)
Goal of FOR*
Raters develop common frame of reference
Observing performance
Evaluating performance
*Most appropriate when PM appraisal system focuses on
behaviors
38. Psyc601 - K. Shultz, Week 6
25
Behavioral Observation Training (BO)
Goals of BO
Minimize unintentional rating errors
Improve rater skills by focusing on how raters:
Observe performance
Store information about performance
Recall information about performance
Use information about performance
Psyc601 - K. Shultz, Week 6
26
Self-leadership Training (SL)
Goals of SL
Improve rater confidence in ability to manage performance
Enhance mental processes
Increase self-efficacy
Psyc601 - K. Shultz, Week 6
27
Pilot Testing
Provides ability to
Discover potential problems
Fix them
Benefits
39. Gain information from potential participants
Learn about difficulties/obstacles
Collect recommendations on how to improve
Understand personal reactions
Get early buy-in
Get higher rate of acceptance
Psyc601 - K. Shultz, Week 6
28
Implementing a Pilot Test
Roll out test version with sample group
Staff and jobs generalizable to organization
Fully implement planned system
All participants keep records of issues encountered
Do not record appraisal scores
Collect input from all participants
Psyc601 - K. Shultz, Week 6
29
Ongoing Monitoring and Evaluation
When system is implemented, decide:
How to evaluate system effectiveness
How to measure implementation
How to measure results
Psyc601 - K. Shultz, Week 6
30
40. Evaluation data to collect
Reactions to the system
Assessments of requirements
Operational
Technical
Effectiveness of performance ratings
Psyc601 - K. Shultz, Week 6
31
Indicators to Consider
Number of individuals evaluated
Distribution of performance ratings
Quality of information
Quality of performance discussion meetings
System satisfaction
Cost/benefit ratio
Unit-level and organization-level performance
Psyc601 - K. Shultz, Week 6
32
Case Study 7-2 and 7-3
After implementing the PM process (via Exercise 7-1)
Setting up an appeals process (Exercise 7-2)
Evaluating the process (Exercise 7-3)
Psyc601 - K. Shultz, Week 5
32
41. Psyc601 - K. Shultz, Week 6
33
Summary – Chapter 6
Several keys to good and useful PA forms
Can combine information via mechanical or holistic approaches
Several practical issues to work out (e.g., appeals period, who
should rate)
Many potential motivators for raters
Several options to reduce rater distortion
Psyc601 - K. Shultz, Week 5
33
Psyc601 - K. Shultz, Week 6
34
Summary – Chapter 7
Implementation of a solid PM process requires lots of
preparation
Rater training a key component
Many options here
Pilot testing and ongoing monitoring keys to success
Psyc601 - K. Shultz, Week 5
34
Psyc601 - K. Shultz, Week 6
35
42. Psyc601 - K. Shultz, Week 5
35
35
35
Next Time
Discussion of Chapter 8 in Aguinis – PM and Employee
Development
Presentations by:
Zytlaly (360 degree feedback), and
Jamie (personal development plans)
35