Callcopy qa-guide

Contact Center QA Guide:
Building a World-Class Quality Assurance Program
Sponsored by:

About DMG Consulting LLC
DMG Consulting LLC is the leading analyst firm in the contact center and analytics
markets. We are a strategic advisor to companies large and small. Our mission is to
help companies build effective contact center and service environments that provide
an outstanding customer experience. We achieve this goal by helping our clients
leverage technology, process and people to optimize operational efficiency, sales and
profits. Our actionable strategies and tactics effect change that enable companies to
make strategic and tactical improvements with a rapid return on investment. Our
customers include Global 2000 companies, government agencies, non‐profit
organizations, and more than 150 contact center and analytics vendors.

DMG Consulting LLC’s business strategists have an average of 20 years experience in
customer experience management (CEM), customer relationship management (CRM),
contact centers, building customer‐focused businesses and financial analysis. We
understand the power of customer data and the contributions that quality
management, recording, speech analytics, performance management, surveying,
eLearning, coaching and workforce management systems make to the contact center
and the entire enterprise.

Our consulting experience with more than 2000 end‐user organizations and our hands‐
on operational, technology and financial expertise give us deep insight into what
customers want and need from enterprises and vendors.
All rights reserved.

This report is protected by United States copyright law and may not be copied, reproduced,
displayed, published, transmitted or distributed in whole or in part in any form or medium
without the express written permission of DMG Consulting LLC. You may not alter or remove
any trademark, copyright or other notice from this report.

Reproduction in whole or in part in any form or medium without express written permission of
DMG Consulting LLC is prohibited. Substantial effort went into verifying and validating the
accuracy of the information contained within this Report, however, DMG Consulting LLC
makes no warranties as to the accuracy or completeness of this information. DMG Consulting
LLC is not liable for any damages, consequential or otherwise, arising from use of this
information.
Copyright © 2009 DMG Consulting LLC
© 2009 DMG Consulting LLC - i - May 2009

Table of Contents
Executive Summary.....................................................................................................1
1. Introduction .....................................................................................................2
2. What is Contact Center Quality Assurance? ......................................................3
3. Quality Assurance Program Benefits ................................................................6
4. Components of a QA Program..........................................................................7
5. Getting Started ..............................................................................................10
5.1 Creating a Quality Assurance Leadership Team..............................................15
5.2 Staffing QA Positions .....................................................................................16
5.3 Implementation Roadmap..............................................................................18
6. Developing a QA Evaluation Form ..................................................................21
6.1 Form Categories.............................................................................................24
6.2 QA Evaluation Form Questions.......................................................................25
6.3 Assigning Weights to QA Evaluation Forms....................................................26
6.4 Validation.......................................................................................................27
6.5 Example QA Evaluation Forms .......................................................................29
7. Quality Assurance Monitoring Criteria and Guidelines ....................................49
7.1 Quality Monitoring Criteria and Guidelines.....................................................49
7.2 Why are Quality Monitoring Criteria and Guidelines Necessary?.....................50
8. Calibration......................................................................................................51
8.1 What is Calibration?........................................................................................51
8.2 Benefits of Calibration.....................................................................................52
8.3 The Calibration Process ..................................................................................53
9. Quality Assurance Program Mechanics and Processes....................................55
9.1 Transaction Selection Criteria.........................................................................55
9.2 Determining the Number and Frequency of Evaluations.................................56
9.3 Coaching ........................................................................................................ 57
9.4 Evaluation Feedback and Escalation Process ..................................................59
9.5 Ongoing Training ...........................................................................................60
9.6 Addressing Agent Performance Issues............................................................61
9.7 Rewards and Recognition ...............................................................................64
9.8 Updating Procedures/Training........................................................................65
9.9 Monitoring Quality Assurance Reviewers.......................................................66
9.10 Reporting .......................................................................................................68
9.11 QA Database.................................................................................................. 75
10. Kicking Off the QA Program...........................................................................76
10.1 Agent Training ............................................................................................... 77
10.2 QA Program Pilot ...........................................................................................79
© 2009 DMG Consulting LLC - ii - May 2009

© 2009 DMG Consulting LLC - iii - May 2009
11. Advanced Quality Assurance Initiatives ..........................................................80
11.1 Surveying .......................................................................................................80
11.2 Customer Experience Monitoring ...................................................................80
11.3 First Call Resolution (FCR)...............................................................................81
12. Quality Management/Liability Recording Suites.............................................82
Appendix A: Procedure Format Sample .....................................................................86
Appendix B: Policy Format Sample ............................................................................89
About CallCopy..........................................................................................................90

© 2009 DMG Consulting LLC - iv - May 2009
Table of Figures
Figure 1:    Quality Assurance Process............................................................................5
Figure 2:    Quality Assurance Program Development Roadmap ..................................10
Figure 3:    QA Implementation Roadmap....................................................................19
Figure 4:   Common Quality Evaluation Form Sections/Categories ..............................24
Figure 5:    Healthcare Customer Service QA Evaluation Form .....................................30
Figure 6:   Technical Support QA Evaluation Form.......................................................34
Figure 7:    Retail Contact Center QA Evaluation Form .................................................38
Figure 8:    Sales Contact Center QA Evaluation Form..................................................42
Figure 9:    Financial Services Customer Service QA Evaluation Form...........................45
Figure 10:  The Calibration Process..............................................................................54
Figure 11:  Coaching Methods ..................................................................................... 57
Figure 12:  Average Contact Center QA Score: December 2008...................................68
Figure 13:  Contact Center Average Scores by Evaluation Section: December 2008 .....69
Figure 14:  Average Scores by Evaluation Question: December 2008........................... 71
Figure 15:  Average QA Scores by Agent: December 2008........................................... 73
Figure 16:  QA Scores by Agent by Evaluation: December 2008................................... 73
Figure 17:  Average QA Scores by Agent by Category: December 2008........................74
Figure 18:  QA Agent Training Outline......................................................................... 77
Figure 19:  Workforce Optimization Suites ..................................................................83

© 2009 DMG Consulting LLC - 1 - May 2009
CCCooonnntttaaacccttt CCCeeenntteerr QQQAAA GGGuuuiiidddeee nter
Executive Summary
Contact center quality assurance (QA) is an ongoing business activity that provides
valuable insights into agent, customer and contact center service delivery performance and
opportunities. The primary goal of any QA program is to verify and ensure that a company’s
contact center services are being performed in a manner that meets or exceeds internal
requirements and customer expectations. At a fundamental level, QA programs measure
how well agents comply with internal policies and procedures and interact with customers in
phone, email and chat sessions. The more advanced QA programs go well beyond these
basics. They combine the results of customer satisfaction surveys with internal
measurements to provide a 360‐degree view of the customer experience. Well‐designed and
effective QA programs demonstrate a company’s commitment to its customers and agents,
and are essential for building world‐class contact centers. This Guide is a “cookbook” for
building a strong, effective and well‐received QA program. It is ideal for managers and
supervisors in contact centers of any size who are either building their first quality assurance
program or want to enhance an existing program.

CCCooonnntttaaacccttt CCCeeenntteerr QQQAAA GGGuuuiiidddeeenter
1. Introduction
Quality assurance isn’t an option for contact centers; it’s essential for the success of the
contact center, customer and agent satisfaction, improving agent and supervisor productivity and
effectiveness, and keeping management in touch with their staff’s performance. To achieve the best
results and foster confidence in the program, managers must ensure that they are evaluating the
right components of agent performance during customer interactions and using appropriate
measurements and weights. Building an effective QA program is an iterative, multi‐step process
that requires senior management support, planning and input, and buy‐in from all levels of contact
center staff. Automation is helpful for formalizing, standardizing and institutionalizing the initiative,
but QA a program will succeed only if the staff is on board and believes in its value.

2. What is Contact Center Quality Assurance?
Contact center quality assurance, also known as Quality Management (QM), is a process
where managers, supervisors and QA specialists monitor and evaluate how well agents handle
customer transactions. The monitoring process includes a simultaneous review of a call and the
system screens used to handle the interaction. (If QA is being done on an email or a chat session, it
will review just the servicing screens and the content of the agent’s response.) QA can be done in a
real‐time mode where the supervisor or QA specialist “live monitors” calls. For live monitoring QA,
reviewers can either access calls through their automatic call distributor (ACD) technology, or can sit
next to agents and watch and evaluate them as they handle calls. QA is often performed on
recorded transactions.

There are pros and cons to both live monitoring and using recorded transactions. When live
monitoring, the QA specialist/supervisor can provide immediate feedback to the agent. This can
become an effective coaching opportunity, as the feedback is provided in real‐time when the call is
still fresh in the agent’s mind. On the other hand, doing QA from recorded transactions allows an
organization to schedule calls, making the process more efficient for the reviewer. It allows QA
specialists/supervisors to find interactions that require their attention, either because they are really
good or really bad, instead of wasting time on satisfactory calls. It allows the reviewer to be able to
carefully review the interaction/screens, including the ability to go back and re‐review a portion of
the interaction. And, lastly, it is an anonymous process, so agents are not aware that they are being
evaluated and will not necessarily be on their best behavior.

When doing QA, the reviewer completes an evaluation form that measures how the agent handled
each component of the call or online transaction. The form can be on paper or part of a QA system.

The fundamental purpose of QA programs is to measure how well agents adhere to contact center
departmental policies and procedures. Contact/call center managers have traditionally live
monitored or listened to recorded interactions in order to gauge agent performance and internally
evaluate the customer experience. To be effective, the program should be a formalized ongoing
process designed to:

1. Measure agent adherence to internal policies and procedures
2. Improve consistency and quality of customer interactions across all
                                channels (telephone, email, chat/IM, etc.)
3. Assess business execution – detect and fix broken or inefficient
                                policies, processes or operational issues throughout the company
4. Improve agent performance
5. Identify agent training needs
6. Identify policies or processes that frustrate and alienate customers
7. Maximize every customer interaction
8. Identify business trends
9. Improve the customer experience

The underpinnings of a good QA program are consistency, accurate measurement, and a cycle of
continuous feedback resulting in improvements. An effective QA program provides the contact
center with a vehicle for measuring the quality and consistency of service delivery, capturing
customer insights, and identifying trends, service issues and training/coaching opportunities to
improve agent performance and productivity. Quality assurance is a dynamic and iterative process
that must be adapted as a business changes. See Figure 1.

CCnter Cooonnntttaaacccttt CCCeeenntteerr QQQAAA GGGuuuiiidddeee
Figure 1: Quality Assurance Process

Source: DMG Consulting LLC, May 2009

Action Item: Identify the company’s reasons for performing contact center quality
assurance. Build a program that delivers continuous feedback to the contact center
and other operating departments.

3. Quality Assurance Program Benefits
When quality assurance programs are well designed and used consistently, they yield great
benefits for customers, enterprises, contact centers and agents. The benefits include:

• Improved agent productivity, by reducing the average handle time of transactions
• Reduced operating expenses
• Better transaction quality
• Increased customer satisfaction
• Enhanced customer experience
• Identification of business opportunities, cross‐sell, up‐sell, new products and services
• Enhanced operating policies and procedures
• Reduced enterprise risk
• Improved agent satisfaction and reduced attrition
• Automated reporting for tracking and trending
Action Item: Set up a process for capturing, quantifying and reporting the benefits
from your QA program. Be sure to share successes with agents, supervisors,
managers and senior executives on an ongoing basis, so that everyone appreciates
the program’s contributions.

4. Components of a QA Program
While every company should customize their QA program to reflect their corporate culture
and values, the fundamental building blocks for contact center QA are standard across all
companies. Effective QA programs should include the following components:

• Procedures and policies: Document all existing transaction procedures and policies so that the
contact center staff knows the appropriate steps for handling all types of inquiries. QA
specialists/supervisors can also use the procedures and policies as a standard to make sure that
they are evaluating different transactions properly. It’s important to set up a process that keeps
the policies and procedures up‐to‐date.

• Quality monitoring criteria and guidelines: Specify the criteria to apply when evaluating
transactions and performing evaluations. The criteria should define what QA
specialists/supervisors are looking for in each type of contact center transaction. The easiest way
to create criteria is to use the contact center’s documented procedures and policies and note on
each the most important aspects for each type of transaction. It’s also a good idea to identify
agent actions that would cause them to lose points in a quality evaluation.

• Program mechanics: Define the mechanics of the QA program, including who (manager,
supervisor, team leader, QA specialist, trainer) is responsible for conducting evaluations, the
number and frequency of evaluations, how many coaching sessions per agent per month, and
how to select calls, emails and chat sessions for evaluation. (If a QA application is being used,
most of the mechanics will be automated, including the transaction selection process.)

• Training: Establish a closed‐loop training process that addresses new content, system issues,
updates and agent performance issues. The trainers and QA staff must work closely together to
ensure that the staff is fully trained. (In many small/mid‐sized contact centers, the same people
do both QA and training.) Prior to kicking off a QA program, all contact center staff – agents,
supervisors, QA specialists, trainers, managers – must be fully trained so that they know how to

handle all types of customer interactions. If new or enhanced procedures and policies are
drafted to support the QA program, they should be reviewed with the staff before starting the
QA program. In addition to agent procedural training, it’s important to build a training program
that introduces the new or enhanced QA program to the staff. The more informed the staff is
about the program, the more effective the initiative.

• Coaching: Provide frequent feedback to agents about their performance. Feedback should
address where agents are performing well and areas where they have opportunities to improve.
Coaching is one of the critical success factors in QA programs and plays a very important role in
agent satisfaction and retention. However, as it can be very challenging to provide negative
feedback to agents, it’s important to train the QA staff to produce effective coaching sessions
and to make sure that management delivers coaching sessions consistently.

• Calibration: Build consistency into the QA program through calibration. Calibration is the
process of teaching all people involved in performing QA evaluations how to score transactions
on a consistent and equitable basis. To make a QA program fair for agents, it’s essential for all
QA reviewers to agree on the meaning and value of each question in a monitoring form. To
achieve consistency, it’s important to run calibration sessions where all reviewers listen to the
same call, score it, identify variance in scoring approaches, reconcile their differences, and set a
standard measurement that all will use going forward. The only way to reach consensus is to run
calibration sessions. Calibration is an ongoing process and should be run on a monthly basis. It’s
also a good idea to involve agents in the calibration process so that they can appreciate the
challenges associated with consistently evaluating transactions.

• Evaluation feedback: Provide a process to facilitate two‐way communication between agents
and reviewers. Agents need to have a mechanism for responding to their quality evaluations so
that they feel empowered and not “put upon.” They also need a formal process for filing
complaints when they believe that a QA reviewer is not being fair or is not listening to their
input. QA reviewers should welcome discourse, as it will ultimately yield a better and more
effective program.


• Rewards and recognition: Recognizing and rewarding top performers is essential for the
success of a QA program. While agents should deliver outstanding performance because it’s
their job, recognizing when they do encourages them to keep up the good work and motivates
others to strive for recognition, as well. Rewards do not have to be large; they could include a
plaque, a parking spot, lunch with the CEO, a gift card, movie tickets, etc.
Action Item: Implement an “Executive QA” program, where senior managers from sales,
marketing, operations and all other supporting areas sit with contact center agents as they
handle calls. This has proven to create customer‐focused awareness and foster collaboration
between departments. (Rapid process change is facilitated when senior executives hear first‐
hand the impact of their processes and programs on customers.) When senior managers take
this program seriously, it has a very positive impact on agent morale and satisfaction
because it underscores the importance of their job.

5. Getting Started
Figure 2 depicts a high‐level project plan for building a QA program. This figure provides the
high‐level steps that each organization should customize to meet the needs of their company. The
first step in the process is identifying and assigning resources to oversee and manage the program
on an ongoing basis. This step is critical because every company needs a champion to push the
program through, or it is unlikely to happen. (The project champion often becomes the manager or
supervisor responsible for the program.) It takes a significant amount of work to build a contact
center QA program and there must be an individual responsible for making sure that everyone
involved in the program is motivated and the project stays on track and on time.

Figure 2: Quality Assurance Program Development Roadmap


Pilot the program for 2 months and review results
Identify QA manager and reviewer staff
Communicate to all staff that a formal Quality Assurance program is under development
Develop the evaluation forms; determine section and question weights
Test the form using real calls, conduct calibration and refine the form as needed
Develop quality monitoring criteria and guidelines
Define the coaching and feedback process
Build Quality Assurance training program
Train all staff members on the Quality Assurance program
1
2
3
4
5
6
7
8
.
Adjust program and repeat pilot as needed
9
10
Implement the formal QA program11
Pilot the program for 2 months and review results
Identify QA manager and reviewer staff
Communicate to all staff that a formal Quality Assurance program is under development
Develop the evaluation forms; determine section and question weights
Test the form using real calls, conduct calibration and refine the form as needed
Develop quality monitoring criteria and guidelines
Define the coaching and feedback process
Build Quality Assurance training program
Train all staff members on the Quality Assurance program
1
2
3
4
5
6
7
8
.
Adjust program and repeat pilot as needed
9
10
Implement the formal QA program11 Implement the formal QA program11

The 11 high‐level steps for building a QA program are explained below. It’s important to note that
departmental procedures for handling all types of customer transactions and inquiries should be
drafted or enhanced prior to developing the QA process.

Step 1: Identify resources from the contact center to serve as the QA manager and quality
review staff. (The QA manager will also play a major role in building or enhancing the program.)
Ideally the manager should have prior experience in setting up and running a QA program and
conducting contact center QA evaluations. Minimally, the QA reviewer(s) must demonstrate
complete and accurate knowledge of all contact center policies, procedures and systems, as well
as having excellent interpersonal and coaching skills. The number of resources dedicated to the
process varies based on the size of the contact center.

Step 2: The institution of a formal QA program should be communicated to all staff members as
early as possible. The success of the program depends upon agent cooperation, which will be
enhanced if agents are invited to participate in program development and all changes are clearly
communicated. Communication should include information about why the program is being
developed, roles and responsibilities, impact on agents (their reviews, raises, and incentives),
and program benefits.

Step 3: Develop the quality evaluation forms. Typically, managers create one for each channel
supported (phone, email, chat), and/or one for different call classifications, i.e. service, sales,
collections, fraud, etc. Once the number and types of forms are decided, determine the sections
required in each form, i.e., opening, verification, problem resolution, communications, closing,
and then write the questions for each section. Once the sections and questions are drafted,
assign weights to each question and possibly to each section, reflecting their relative
importance. It’s easiest to use a 100‐point scale but this is not a requirement.

Step 4: Using real calls, test the evaluation forms to ensure all call components that need to be
measured are captured and that the intended goals are achieved fairly and accurately. Conduct
calibration sessions involving supervisors and QA specialists to ensure that everyone
participating in the QA program is using the form correctly and with the same rigor, consistently

and fairly across all questions. It generally takes a few calibration sessions to finalize the
questions and weights for each evaluation form.

Step 5: Document the quality monitoring evaluation criteria and guidelines for how to apply
them.

Step 6: Define the agent coaching and feedback process. Coaching is considered the most
effective method for training adults; it should take place in a one‐on‐one personalized session
with an agent. During coaching sessions, agents should receive specific and targeted feedback
to learn what they are doing well as well as the areas where they can improve. This is a critical
part of successfully managing agents and motivating them to improve their performance.

Step 7: Build the QA training program. Training should include an in‐depth review of the quality
monitoring process and QA form, evaluation criteria and metrics/measurements, frequency and
number of evaluations conducted, scoring methodology, how to access, review and respond to
evaluations, and how to provide feedback to improve the program. Optionally, the QA training
program may also include a review of documented policies, procedures and guidelines that
govern agents’ behavior in processing transactions or advising customers.

Step 8: Train staff on the QA program. Roll out the training program so all agents and other
relevant contact center staff – managers, supervisors, trainers, QA specialists, agents, and
possibly participants from other departments – become familiar and comfortable with the new
program. If possible, conduct calibration sessions with agents during the training session to
demonstrate application of the quality criteria and use of the evaluation form. (Note: Be sure to
ask the agents whose calls will be used during calibration for their permission.) Agents who
understand how their calls are being evaluated and know what is expected of them are more
likely to perform well.

Step 9: Pilot the QA program and monitor results. The pilot provides an opportunity to see what
works and what needs to be enhanced. It also gives QA reviewers and agents an opportunity to
get accustomed to the program.

DMG recommends a 2‐ to 3‐month pilot to work the kinks out of the process and give everyone
time to get used to the new program before going live.

Step 10: Adjust the program throughout the pilot and, if changes are significant enough, it may
make sense to keep the pilot going longer. This step allows time to refine the program before it
is formally launched. Based on participant feedback and pilot results, the QA manager should
revise processes, evaluation criteria, evaluation forms, metrics, mechanics, etc. The pilot should
be run for as long as necessary to test all components of the QA program, including conducting
calibration exercises.

Step 11: Implement the QA program. Launch the program and begin to execute the QA
processes on a daily basis. QA reviewers should work closely with management, supervisors and
trainers to review quality assurance results and identify areas for training and development.
Periodic reviews to evaluate the QA process should be conducted to ensure compliance with
ever‐evolving procedures, policies and protocols. It is also a good idea to establish a reward
program or agent appreciation events, so that management can show special recognition for
outstanding employee performance based on QA program reviews.

After the QA program is implemented, contact centers should consider doing the following activities
on a monthly basis. They are not a requirement but will help to enhance agent moral and improve
the results of your QA program.

• Conduct monthly calibration sessions with all supervisors/QA reviewers to ensure program
consistency.

• Conduct monthly agent team meetings to review QA results and do group training, as
necessary.

• Have QA reviewers and trainers meet on a monthly basis to review training needs and
other improvement opportunities uncovered in the QA process. (In many contact centers,
the QA specialists and trainers speak daily.)

• Publish a monthly QA newsletter (or other vehicle) with quality tips to help the
department achieve quality goals. The newsletter should also recognize top performers.

Action Item: Implement a process where top‐performing agents conduct side‐by‐side
training sessions with agents who are not meeting their potential and are “under ‐
performing”. Agents learn best from their peers. Using the top performers for this activity will
recognize their outstanding performance and help get them on board in support of
management objectives.

5.1 Creating a Quality Assurance Leadership Team
Building an effective QA program is an iterative process that requires senior management
support, in addition to thorough planning, input and buy‐in from all levels of contact center staff. It’s
a best practice to form a QA leadership team with representation from all contact center
constituents including managers, supervisors, trainers, agents, and possibly other departments,
such as marketing and sales. This team could be led by the head of QA or the contact center
director. This is a good way to keep the head of the contact center engaged in the QA program on an
ongoing basis.
Action Item: Use the quality assurance monitoring process to identify areas for business
process optimization. Establish a cross‐functional team to address contact center and
enterprise business process opportunities identified during the quality monitoring process.
This team should work together to change business processes that upset customers.

5.2 Staffing QA Positions
The success of any QA program is largely dependent upon the commitment and skills of the
quality assurance specialist who administers the program and coaches agents. Quality assurance
specialists must be highly skilled individuals who possess outstanding job knowledge and excellent
communication, interpersonal and coaching skills. Because a vast amount of job knowledge is
required for this role, most contact centers staff these positions by promoting agents who
demonstrate exceptional job knowledge and call handling skills. (Other contact centers transfer a
supervisor or manager into this position.) However, being an outstanding customer service or sales
representative is not the same as knowing how to deliver effective one‐on‐one feedback to coach
and motivate a contact center agent. The ability to coach is not innate, and generally has to be
taught. QA specialists should have the following skills and knowledge:

• Job knowledge: To accurately, effectively, and objectively evaluate agents, quality
assurance specialists must possess strong knowledge of all products, services, systems,
processes and procedures. If quality assurance specialists are hired externally, they should
be required to complete a comprehensive training course that includes in‐depth coverage
of all products, services, policies procedures and guidelines that agents are required to
adhere to when processing transactions or advising customers.

• System knowledge: Quality assurance specialists must be fully trained and knowledgeable
about all systems that agents use to handle customer inquiries or process orders.
Additionally, they must be fully trained to navigate the quality management application so
that they can search, retrieve and play back calls, create evaluations, and respond to
agents’ evaluation feedback. QA specialists should also be trained to access and use
reports from the quality application as well as to create ad hoc reports.

QA program mechanics: Quality assurance specialists must possess a strong understanding
of the quality monitoring process, including how to select calls/emails/chat sessions, the
number/frequency of evaluations that they are required to do, and how to coach agents.

• QA criteria/calibration: To maintain measurement integrity, it’s essential for all QA
specialists to uniformly apply quality monitoring criteria when evaluating calls. All QA
reviewers should be thoroughly trained to complete evaluations in a consistent manner.
The success and effectiveness of the program and its reception by agents depends upon
the ability to consistently, objectively and fairly evaluate contact center interactions. To
achieve these essential goals, it’s also recommended that calibration sessions be
conducted with all quality reviewers on a monthly basis.

• Coaching/motivation: QA specialists must demonstrate strong interpersonal and coaching
skills so that they can work one‐on‐one with agents to recognize areas of strengths as well
as provide directed feedback on areas that require improvement. All quality assurance
specialists should be required to take courses in effective coaching methods and
motivation techniques.
Action Item: When staffing QA programs, use highly respected staff members with strong
product, service, system knowledge and expertise in coaching when staffing QA programs.
Agents are more welcoming of feedback from people they respect.

5.3 Implementation Roadmap
Once the leadership team is established and the quality manager and staff have been
appointed, a detailed breakdown of all program deliverables should be developed and assigned to
the responsible parties. Figure 3 provides a list of the required tasks, recommendations for
ownership of each initiative, and estimated time frames for completion. The tasks reflected in Figure
3 have already been discussed in other sections of this Guide. They are repeated here in order to put
them into a project plan and to show the responsible party and estimated time frame.

DMG recommends using this list of initiatives to build a detailed project plan. Your project plan will
likely include other steps and initiatives that are important for your organization. Once the project
plan is drafted, review it and get the buy‐in for all involved parties, particularly those assigned tasks,
to make sure that they are committed and able to dedicate time to the projects. It’s also a good idea
to have a weekly project review meeting to assess progress and to address items that are slipping. It
is ideal, but not necessary for senior contact center management to participate in the weekly
progress meeting. However, if a contact center manager is not able to participate in the weekly
meetings, the project manager should make an effort to keep him/her updated on the team’s
progress.

Figure 3: QA Implementation Roadmap
Task/Initiative Responsibility Timeframe Status
Development Phase
Communicate to all staff that a quality assurance
program is under development and will soon be
implemented; communication should be frequent
Contact Center
Director
2 days
Promote or hire a QA specialist and quality
reviewers
Contact Center
Director
3 weeks
Develop a call monitoring evaluation form (Note:
In multi-channel contact centers, a QA form will
need to be created and tested for each channel)
Quality Assurance
Leadership Team
2 weeks per
form
Test the form using real calls to ensure that all
call elements that need to be evaluated are
captured
QA Manager and
reviewers
2 days
Document quality monitoring evaluation criteria
and guidelines for how to apply them
Quality Assurance
Leadership Team
2 weeks
Determine weights for each section of the
monitoring form
Quality Assurance
Leadership Team
1 week
Test the form(s) using real calls to validate
sections, questions, weights and ranges
Quality Assurance
Leadership Team
1 week
Hold calibration sessions with agents, supervisors
and managers to make sure everyone uses the
evaluation form the same way
Quality Assurance
Leadership Team
2 weeks
Enhance QA evaluation form based on input from
calibration team
QA Manager and
reviewers
3 days
Determine the volume of transactions (calls,
emails, chat sessions, other) to be monitored per
agent/month
Quality Assurance
Leadership Team
1 day
Decide who will be conducting the evaluations
(quality assurance specialists, supervisors, or
both)
Quality Assurance
Leadership Team
1 day
Conduct calibration sessions to ensure rating
reliability and consensus of all reviewers
Quality Assurance
Leadership Team
2 weeks
Define the agent coaching process QA Manager, contact
center managers,
supervisors and
reviewers
1 day
Develop a process with an escalation/review
procedure for agents to provide feedback or
dispute evaluations
Quality Assurance
Leadership Team
1 day
Develop reports to support the QA process; if
using a QA application, build the reports in the
application
Quality Assurance
Leadership Team
2 weeks
Test reports QA Manager 1 week

Figure 3: QA Implementation Roadmap
Task/Initiative Responsibility Timeframe Status
Development Phase
Train all QA specialists to use the QA application
and reports
QA Manager 2 days
Develop QA training program QA Manager and
trainers
3 weeks
Train all contact center staff on the quality
assurance program
QA Manager and
trainers
Variable
depending on
size of staff
and program
length
Kick off the program Contact Center
Director/QA Manager
1 day
Pilot program for 2 to 3 months and review results Quality Assurance
Leadership Team
2 – 3 months
Revise departmental Agent Performance
Evaluation form to include quality monitoring
metrics
Quality Assurance
Leadership Team/HR
1 week
Implement QA program Contact Center
Leadership Team
On-going
Establish a process for identifying and
recognizing agents who achieve quality
monitoring excellence
Contact Center
Leadership Team
2 weeks
Perform monthly calibration sessions QA Manager, all QA
reviewers
Monthly/
on-going
Conduct monthly training sessions Trainer Monthly/
on-going
Create and issue monthly QA newsletter QA Manager Monthly/
on-going
Have QA staff meet with training to enhance
training programs
QA Manager and
trainers
Monthly/
on-going

Action Item: Involve all levels of contact center staff in creating the program to avoid
unnecessary skepticism and surprises. This will help agents appreciate the positive aspects
of the QA program and speed up adoption.

6. Developing a QA Evaluation Form
When managers think about developing a QA program, one of the first things they must
consider is creating a QA evaluation form. The evaluation form is the most visible component of the
program and is necessary for all QA programs, whether an organization is doing QA manually or
using an automated system.

The QA evaluation form needs to capture all interaction components that a contact center wants to
measure. Its questions and weights should reflect the culture of the company and what is most
important to their service strategy. In general, QA evaluation forms contain the following
components:

1. Call/evaluation details
Examples: Name of agent, date of transaction, reviewer name, date of evaluation, call type,
customer identifier (account number, social security number, etc.)

2. Sections (skill categories)
Examples: Call opening/closing, verification, product/plan knowledge, procedure
knowledge, system knowledge, hold/mute/transfer, communication skills, resolution skills,
etc.

3. Questions (to objectively assess skill proficiencies)
Examples: Did the representative identify the caller according to the verification policy prior
to releasing information? Did the representative log a summary of the call according to the
policy?

4. Scoring
Point values for each question and section of the evaluation form; includes point values,
points available, and points earned. Point values should be assigned based on the relative
importance of each section and question to the business and the customer.

5. Coaching comments
Free‐form text box to allow reviewers to provide feedback on performance excellence or
opportunities.

6. Recommendations
Free‐form text box to allow reviewers to document an action or follow‐up items for agents
to complete. Examples: taking a specific eLearning course, reviewing specific procedures,
recommendations, etc.

7. Acknowledgement (optional)
Signature boxes so that the reviewer and agent can sign off and acknowledge that they had
a discussion.

After learning how to create a QA evaluation form and deciding what sections are required, it’s
relatively easy to modify an existing form or draft a new one. However, creating one for the first
time can be daunting. Therefore, we suggest getting started with an existing QA form and then
modifying it to meet the organization’s specific needs. The easiest way to obtain sample quality
evaluation forms is by asking a QA vendor, another contact center manager, or a consulting firm.
(Most vendors have sample forms and are generally happy to share them, particularly if they think
you are interested in purchasing their solution.)

While it’s relatively easy to obtain a sample QA evaluation form, the catch is that there is no "one
size fits all" form. For example, a technical help desk needs a different QA evaluation form from a
sales or customer service contact center. The QA evaluation form should be customized to meet the
needs of each particular contact center.

Here are a few best practices to employ when developing QA evaluation forms:

Involve all levels of your staff – agents, supervisors, trainers and managers.

Develop a separate QA form for each channel that you will be monitoring. Test
your QA form(s) using real transactions and refine them before putting them into
production.

Assign weights that correspond to the importance of all categories on your QA
form. Importance may be relative to the channel of communication, customer
and/or the business.

Include a comments box for each section of the evaluation so that coaching
comments on performance strengths and opportunities and agent feedback can
be captured.

Action Item: To modify or create a QA evaluation form, start with an existing or sample
form. Then listen to a selection of different call types. While “scoring” these calls, identify
form components that need to be changed or added to fit your environment. (This
process applies to emails, chats or other types of transactions received by the contact
center.)

CContCoonnttaaacccttt   CCCeeennnttteeerrr   QQQAAA   GGGuuuiiidddeee
6.1 Form Categories
Figure 4 provides an example of sections or categories typically found in a QA form for calls.
It also includes example criteria that can be used to evaluate if an agent properly performed a
particular skill. Depending on how the form is structured, sections can be established based on skills,
call flow, competencies, or call segments. In Figure 4 below, the sections are based on skills.

Figure 4: Common Quality Evaluation Form Sections/Categories


Action Item: Be sure to include a section in the evaluation form to reflect how well the
agent performed in regard to compliance or disclosure regulations (if applicable).

Quality Evaluation Form
Call Opening/Closing
Is there a policy/procedure/service issue that prevented First Contact Resolution?
Demonstrated thorough knowledge of product/part/services/warranty, etc.
Effectively accessed information on system; utilized all appropriate systems to obtain information
Use hold/transfer effectively and only as necessary
Did the agent do everything possible to ensure a callback was not necessary?
Spoke clearly and confidently
Accurately diagnosed problem/issue
Obtained verification prior to releasing information according to established data security procedures
Provided name/company name; thanked the customer for calling or purchasing [brand] products
Is there a policy/procedure/service issue that prevented First Contact Resolution?
Demonstrated thorough knowledge of product/part/services/warranty, etc.
Effectively accessed information on system; utilized all appropriate systems to obtain information
Use hold/transfer effectively and only as necessary
Did the agent do everything possible to ensure a callback was not necessary?
Spoke clearly and confidently
Accurately diagnosed problem/issue
Obtained verification prior to releasing information according to established data security procedures
Provided name/company name; thanked the customer for calling or purchasing [brand] products
Demonstration of System Knowledge/Usage
Demonstration of Product Knowledge and Information
Hold/Transfer Procedure
Verification
Demonstration of Resolution Skills
Communication Skills
Agent First Contact Resolution (FCR)
Business Process FCR

6.2 QA Evaluation Form Questions
After identifying the right sections/skills to include in the QA form, the next step is to create
a concise list of questions that captures the elements associated with demonstrating competency in
each skill. The goal is to write questions that allow QA reviewers to completely and objectively
assess agent performance.

Once the initial list of questions is developed for each section of the form, have a team of supervisors
and QA reviewers test the form by trying it out on a few calls or emails. It will typically take at least
ten rewrites of the questions before an evaluation form is complete. This exercise helps surface gaps
and identifies overlaps and redundancies in questions. Revise the form based on test results, and
retest it until the form successfully addresses all requirements. (You will know the form works when
there is a place to address every aspect of a call or email.)

Action Item: Test the QA form with real calls or emails. Make sure there is a place on the
form to address everything that can happen in a call or email. This will surface confusing or
ambiguous components of the form.

6.3 Assigning Weights to QA Evaluation Forms
Once the form content is finalized, it’s time to select a scheme for assigning weights to the
questions and/or sections of the evaluation. The simplest approach is to go with a 100 point scale.
While there are many scoring methodologies, the most basic way is to assign a point value to each
section on the form and distribute the points among the questions in that section. For example, the
communications skill section may be assigned a total of 20 points, which are distributed among the
four questions in the section. While points can be distributed equally throughout a section or the
entire QA evaluation form, we suggest that they be assigned based on the relative importance of
each section and each question. Importance may be relative to the channel of communication,
customer and/or the business. After preliminary weights are distributed, the form should be tested
again by evaluating and scoring actual transactions. Points are generally reconsidered and
redistributed during testing. It’s also important to consider that in some interactions, not all
components of the form are applicable (for example, not every call includes a hold or transfer). The
scoring on the forms shown in Section 6.5 was designed to include “Not Applicable” (N/A) as a
consideration; if a particular question on the form is not applicable to the interaction being
evaluated, the question is marked N/A. The points are not available to be achieved, nor do they
affect the total score. Overall evaluation scores are calculated as follows:

Total points achieved ⁄ Total points available = QM evaluation score
For example: 76 points achieved / 84 points available = 90.4%

Action Item: When assigning weights to the form, it’s also a good idea to identify and define
the serious errors that will result in a full and automatic failure of an evaluation form section
or an entire evaluation.

   CCCooonnntttaaacccttt   CCCeenntteerr   QQQAAA   GGGuuuiiidddeeeenter
6.4 Validation
Validation of the metrics associated with each category of the quality assurance form is an
important step in developing the quality assurance program. Metrics are typically validated by
testing the proposed weights against baseline quality measurements that have already been
established.

When modifying an existing QA evaluation form, the new form can be validated by using it to re‐
evaluate transactions that had previously been scored using a different QA form. If no quality
measurement baseline exists, initial testing sessions should validate QA scores based on agent
performance. So, top performers should receive high scores, average performers should be in the
mid‐range, and poor performers should have low scores.

Once the point values are validated and coming out as expected, the next step is to determine how
QA evaluation scores equate to the performance ratings used by the organization in its annual
review process. For example:

Excellent: 89% and above
Good: 80‐88%
Average: 71‐79%
Needs Improvement: 70% and below

The scoring ranges above are typical for many contact center QA programs. The ranges should be
high enough so that an excellent transaction is clearly identifiable, but not so high so that few agents
can attain an excellent rating. As a result, it’s often a good idea to start with one range and then as
agents get accustomed to the QA program and their quality improves, tighten the ranges. So, for
example, when a QA program is first kicked off, excellent may be anything above 85%. However,
since most customers are not likely to think that 85% is excellent, the range should be narrowed to
anything above either 89% or 90% a few months into the program.

Be sure to communicate to agents exactly what is happening with the ranges – starting easy to give
them a chance to become accustomed, and then bringing them to a level that will better meet
customer expectations – so that the staff does not think that management is just giving them a hard
time. Be sure to set expectations and communicate clearly at all stages of the program to get the
staff’s support.

Once the final program is rolled out, agents should have a two‐ to three‐month grace period to
become accustomed to the program, criteria and scoring before they are held accountable for QA
scores; this will minimize any claims of “unfairness” within the shop, as the staff will have had an
extended period of time to adjust to the program requirements. During this period, QA evaluations
and coaching sessions are performed, but are not counted toward agents’ annual evaluations.

Action Item: Review and enhance your QA evaluation forms periodically —minimally, every
9 to 12 months to keep them in sync with your business and customers’ expectations.

6.5 Example QA Evaluation Forms
This section includes 5 example QA forms; all were provided by CallCopy and are included in their
library of forms. The first form, Figure 5, is a generic QA evaluation form intended for an inbound
customer service contact center. Figure 6 is targeted for a healthcare contact center. Figures 7, 8, 9
are designed for technical support, retail, and travel services contact centers, respectively. Use these
forms as examples for getting started, but customize them to reflect the needs and priorities of your
organization.

Figure 5: Customer Service QA Evaluation Form
Description: Though intended for use by an inbound customer service organization, this form can be
used by many verticals.

Customer Service Evaluation Details
Agent Name:
Call Date:
Evaluator Name:
Evaluation Date:
Section 1: Greeting (5 points)
Point Values Score
Used approved greeting 3 - Yes
0 - No (Auto-fail section)
Tone of voice was upbeat and professional 2 - Exceeds Expectations
1 - Meets Expectations
0 - Fails Expectations
SubtotalComments:
Section 2: Account Management/System Usage (15 points)
Point Values Score
Verified caller and customer account
information correctly
5 - Yes
0 - No (Auto-fail form)
NA
Efficiently used tools properly 5 - Exceeds Expectations
NA
Updated account information correctly as
needed
5 - Yes
0 - No
SubtotalComments:
Section 3: Issue Resolution (32 points)
Point Values Score
Identified customer’s issue quickly 3 - Yes
0 - No
NA
Correctly resolved customer’s issue(s) 10 - Yes
NA

Followed correct call escalation process 3 - Yes
0 - No
NA
Used proper hold procedures (asked
customer to be placed on hold, checked back
in regular intervals, thanked customer for
holding)
3 - Yes
0 - No
NA
Transferred call to the appropriate department
or representative
3 - Yes
0 - No
NA
Effectively explained resolution to customer 5 - Exceeds Expectations
NA
Provided proper alternatives 2 - Exceeds Expectations
Set correct expectations for follow up as
promised
3 - Yes
0 - No
NA
SubtotalComments:
Section 4: Product Knowledge (15 points)
Point Values Score
Demonstrated thorough knowledge of the
product
5 - Yes
0 - No
NA
Gave accurate information 10 - Yes
0 - No
SubtotalComments:

Section 5: Communication Skills/Soft Skills (26 points)
Point Values Score
Maintained a positive tone throughout the call 3 - Exceeds Expectations
Avoided using jargon 2 - Exceeds Expectations
Effectively controlled the call 3 - Exceeds Expectations
Demonstrated professional call etiquette
(please, thank you)
3 - Exceeds Expectations
Communicated at the caller’s level 3 - Exceeds Expectations
Demonstrated active listening skills 3 - Exceeds Expectations
Avoided rude behavior 5 - Yes
Demonstrated confidence 2 - Exceeds Expectations
Appropriately and effectively expressed
empathy and sympathy
NA
SubtotalComments:
Section 6: Closing (7 points)
Point Values Score
Thanked customer for calling 2 - Yes
0 - No
NA
Asked for additional issues 2 - Yes
0 - No
Used approved closing 3 - Yes
NA
SubtotalComments:

Section 7: Notes
Agent Strengths:
Agent Weaknesses:
Agent Recommendations:
Form Total:
Acknowledgements:
Agent:
Evaluator:
Source: CallCopy, Inc.

Figure 6: Healthcare QA Evaluation Form
Description: This form is for use by an inbound customer service organization that handles insurance
claims. The calls may address program eligibility, claims status and dipositioning, program benefits
and coverage.

Healthcare Evaluation Details
Agent Name:
Call Date:
Evaluator Name:
Evaluation Date:
Point Values Score
SubtotalComments:
Point Values Score
Verified caller identity correctly 5 - Yes
NA
Efficiently used tools 5 - Exceeds Expectations
NA
needed
5 - Yes
0 - No
SubtotalComments:

Point Values Score
0 - No
NA
NA
0 - No
NA
holding)
3 - Yes
0 - No
NA
or representative
3 - Yes
0 - No
NA
NA
promised
3 - Yes
0 - No
NA
SubtotalComments:
Point Values Score
product
10 - Yes
0 - No
NA
SubtotalComments:

Point Values Score
(please, thank you)
NA
SubtotalComments:
Point Values Score
0 - No
NA
NA
SubtotalComments:

Section 7: Notes
Agent Strengths:
Agent Weaknesses:
Form Total:
Acknowledgements:
Agent:
Evaluator:

Figure 7: Technical Support QA Evaluation Form
Description: This form is intended to be used by an inbound customer service organization that
provides technical support. Inquiries may address a wide range of issues such as parts, service,
general product information, authorized service repair shops, warranties, detailed technical support,
troubleshooting, etc.

Technical Support Evaluation Details
Agent Name:
Call Date:
Evaluator Name:
Evaluation Date:
Point Values Score
SubtotalComments:
Point Values Score
Verified caller identity correctly 5 - Yes
NA
Efficiently used tools 5 - Exceeds Expectations
NA
needed
5 - Yes
0 - No
Documented issue and resolution (or next
steps) in CRM appropriately
5 - Yes
NA
SubtotalComments:

Point Values Score
0 - No
NA
NA
0 - No
NA
holding)
3 - Yes
0 - No
NA
or representative
3 - Yes
0 - No
NA
NA
promised
3 - Yes
0 - No
NA
SubtotalComments:
Point Values Score
product
10 - Yes
0 - No
NA
0 - No
SubtotalComments:

Point Values Score
(please, thank you)
NA
SubtotalComments:
Point Values Score
0 - No
NA
NA
SubtotalComments:

Section 7: Notes
Agent Strengths:
Agent Weaknesses:
Form Total:
Acknowledgements:
Agent:
Evaluator:

Figure 8: Retail Contact Center QA Evaluation Form
Description: This form is to be used by an inbound retail contact center.

Retail Evaluation Details
Agent Name:
Call Date:
Evaluator Name:
Evaluation Date:
Point Values Score
SubtotalComments:
Point Values Score
5 - Yes
NA
Efficiently used tools properly 5 - Exceeds Expectations
NA
needed
5 - Yes
0 - No
NA
SubtotalComments:
Point Values Score
0 - No
NA
NA
0 - No
NA

holding)
3 - Yes
0 - No
NA
or representative
3 - Yes
0 - No
NA
NA
promised
3 - Yes
0 - No
NA
SubtotalComments:
Point Values Score
product
5 - Yes
0 - No
NA
Stipulated payment terms correctly; stated
refund policy
5 - Yes
0 - No
NA
0 - No
SubtotalComments:
Point Values Score
(please, thank you)

Appropriately and effectively expressed empathy
and sympathy
NA
SubtotalComments:
Point Values Score
0 - No
NA
0 - No
Used approved closing, referring customer to
website
3 - Yes
NA
SubtotalComments:
Section 7: Notes
Agent Strengths:
Agent Weaknesses:
Form Total:
Acknowledgements:
Agent:
Evaluator:

Figure 9: Travel QA Evaluation Form
Description: This form is for use by an inbound contact center that addresses travel‐related inquiries
such as bookings, accommodations and/or services provided.

Travel Evaluation Details
Agent Name:
Call Date:
Evaluator Name:
Evaluation Date:
Point Values Score
SubtotalComments:
Point Values Score
5 - Yes
NA
Efficiently used tools properly - quickly
located
NA
needed
5 - Yes
0 - No
NA
SubtotalComments:

Point Values Score
0 - No
NA
NA
0 - No
NA
holding)
3 - Yes
0 - No
NA
or representative
3 - Yes
0 - No
NA
NA
promised
3 - Yes
0 - No
NA
SubtotalComments:
Section 4: Product and Services Knowledge (15 points)
Point Values Score
product
5 - Yes
0 - No
NA
Correctly explained payment and billing
policy
5 - Yes
0 - No
NA
Correctly explained cancellation policy,
stipulated proper time frames and fees
5 - Yes
0 - No
NA
SubtotalComments:


Point Values Score
(please, thank you)
NA
SubtotalComments:
Point Values Score
0 - No
NA
Directed customer to check website for
updates
2 - Yes
0 - No
0 - No
NA
SubtotalComments:

Section 7: Notes
Agent Strengths:
Agent Weaknesses:
Form Total:
Acknowledgements:
Agent:
Evaluator:

7. Quality Assurance Monitoring Criteria and Guidelines
Quality assurance monitoring guidelines specify the criteria to be used by QA specialists and
reviewers in determining if an agent properly demonstrated a skill during a transaction. They also
indicate which section of the form to use when scoring the various components of the interaction.

7.1 Quality Monitoring Criteria and Guidelines
The quality monitoring criteria define sections, questions and skills that are captured in the
quality evaluation form. They describe how a skill should be demonstrated so that QA reviewers
know how to objectively evaluate and score an interaction. For example, if a question on the
evaluation form is intended to determine if an agent established rapport with a customer, the
criteria must define what building customer rapport means, i.e., addressing the customer by name
throughout the call or acknowledging an event that the customer mentioned in the course of the call
(anniversary, birthday, vacation plans, etc.). Please keep in mind that the meaning of building a
rapport at one company may be the opposite in another. While some companies may want their
customers to call their customers by their first name, others may insist that only a title and the last
name be used.

The criteria should also reflect which section and questions in the QA evaluation form should be
used to score the various call elements. While this may seem obvious, it is often quite difficult.
Consider this example: A quality monitoring form has the following three questions in the
communication skills section: Demonstrated effective listening skills? (3 points), Expressed empathy
and concern as appropriate? (1 point), and Established rapport with the customer? (2 points). If
during the course of a call a customer mentions that they were recently hospitalized and the agent
does not acknowledge that, where should this issue be captured and scored in the prior three
questions? The answer is that as long as it is consistently addressed in the same question, it doesn’t
matter. The quality monitoring criteria must specify where the issue needs to be addressed.

Action Item: Assign a member of the quality assurance review team to be responsible for
overseeing the quality monitoring criteria and maintaining them, as required.

7.2 Why are Quality Monitoring Criteria and Guidelines Necessary?
Quality monitoring criteria and guidelines take the “guesswork” out of figuring out the right
way to evaluate and score a transaction. The guidelines standardize the use of QA evaluation forms,
allowing QA reviewers to handle each transaction objectively and consistently. They also safeguard
against “double‐dipping” – a term used by agents to refer to a situation where they have lost points
for more than one question on a QA evaluation for the same issue. These issues are flushed out and
resolved during the testing and calibration process.

Action Item: Invest time up front to develop clear and concise quality monitoring criteria and
guidelines. This yields a standardized approach and helps minimize misunderstandings about
how to conduct QA evaluations.

8. Calibration
Calibration, a method for building consensus and delivering a standardized evaluation tool,
is key to the success of all QA programs. It is a process intended to ensure that QA results are valid
and based on reliable measurement tools. For a QA program to be credible and produce accurate
and dependable results, it’s essential that all reviewers evaluate transactions on a consistent basis.

Calibration is not a one‐time event. To keep all evaluators synchronized, calibration must be done on
an ongoing basis and include all people involved in conducting evaluations. It’s also a great way to
build camaraderie and support for a QA initiative. Calibration should be conducted on a monthly
basis. If a contact center is a multi‐site environment, all sites should participate in group‐wide
calibration sessions to ensure that transactions are being evaluated consistently regardless of the
site where the transaction is handled.

8.1 What is Calibration?
Calibration is a process where all QA reviewers discuss how to score various types of
transactions. The QA reviewers meet and review agent transactions. Every individual scores the
same transactions and then scoring differences are identified. The reviewers then discuss the
reasons for the differences and reach consensus. DMG recommends that agents be invited to
participate in calibration sessions so that they can gain an appreciation of the effort and rigor
applied to the QA process. When conducted properly, calibration sessions foster collaboration and
establish consensus on how quality monitoring criteria should be applied to each question and
scored on the evaluation form.
Action Item: Update quality monitoring criteria based on the consensus reached in
calibration sessions.

8.2    Benefits of Calibration
When performed consistently, generally monthly, calibration is a proven approach for maintaining
program integrity and equity, and ensuring that agents are treated fairly. Among its many benefits,
calibration:
• Helps develop effective QA evaluation forms
• Teaches QA reviewers how to apply evaluation criteria and perform QA evaluations on a
consistent basis
• Builds consensus among all QA reviewers
• Helps maintain an open dialogue between the QA team and management
• Keeps staff updated about changes to the program, scoring criteria and the evaluation form,
as the program evolves
• Enhances agent perception of the program’s credibility and fairness
• Fosters collaboration and camaraderie among QA reviewers
• Keeps supervisors well versed in how agents are being evaluated and facilitates more
effective coaching

CCCooonnntttaaacccttt CCCeeenntteerr QQQAAA GGGuuuiiidddeee nter
8.3 The Calibration Process
Figure 10 provides a general overview of how calibration works in most contact centers,
although this process varies among companies. A set of recorded calls (and/or emails and chats in a
multi‐channel environment) are tagged for calibration and sent to all QA reviewers. Each reviewer
evaluates and scores the transactions. The QA manager generates a report that reflects scoring
variances between the reviewers for each transaction; this must be done at the question level to be
useful. (In some organizations, the scoring variances are calculated against a “master evaluation”
that is designated as the department standard.) A calibration meeting is held to review scoring
results and to discuss the variances. If a scoring discrepancy is uncovered, each reviewer presents a
justification of their scoring. In most cases, this generates a lively discussion that requires the group
to replay the transaction, or verify information in the procedure or training guide. Once all opinions
are presented, the group reaches a consensus on the best way to score each question. This is likely
to address how the question was rated and/or which form category should be used to capture a
particular issue. This is an important part of the calibration process, as point values vary depending
on where call elements are scored. The quality monitoring criteria guidelines (as well as procedure or
training manuals, if applicable) should be modified or updated based on the consensus outcome.

Figure 10: The Calibration Process

Action Item: Include agents in calibration sessions. This helps them appreciate the effort
management puts into accurately assessing calls and emails and fairly evaluating agent
performance.

9. Quality Assurance Program Mechanics and Processes
An important part of every QA program is determining the number of evaluations that must
be completed to come up with either operationally or scientifically valid results. Once this number is
determined, management must decide if it can afford the resources required to conduct the optimal
number of evaluations. If the answer is “no,” as it is in many contact centers, then the challenge is to
come up with a lower number of evaluations to perform that will still provide insight into agent
performance and transaction trends. Determining the number and frequency of evaluations is called
defining the program mechanics.

9.1 Transaction Selection Criteria
Identifying transactions for QA is an important part of the process. There are four primary methods
for selecting transactions to be reviewed. The first approach is to set up a schedule to capture
transactions at pre‐defined intervals. The second approach is to randomly capture transactions. The
third approach is to use business rules to identify calls that require attention. The fourth method is to
use automation, such as speech or screen analytics, to identify transactions that are either really
good or really bad and require management attention. Regardless of the approach used, it’s
essential that the method capture transactions fairly and equitably. The capture method should be
communicated to agents so that they know how transactions are selected for review.

Once transactions are captured, they can be further qualified based on any number of factors,
including call direction, call duration, call type (based on wrap‐up, disposition or other interaction
classification mechanism), product type, etc. Date of interaction is another key factor as, depending
on the total number of evaluations being conducted per agent/per month, it’s a best practice to
spread out the evaluations over the course of the month. This helps to ensure that agents receive
timely feedback and gives them an opportunity to make adjustments to their performance.
Action Item: Set up a process to capture the best transactions for the QA team.

Callcopy qa-guide

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Callcopy qa-guide

Ähnlich wie Callcopy qa-guide (20)

Callcopy qa-guide