Ian Newcombe from RBS's presentation at our May 2013 Building Capability Conference. 'The role of integrated online assessment in high volume recruitment'
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Building Capability 2013 - Ian Newcombe RBS
1. 1
The role of integrated online assessment in high volume
recruitment
Ian Newcombe C.Psychol, C.Sci, AFBPsS
Head of Assessment
Ian.Newcombe@rbs.co.uk
Objectives
Group Talent at RBS
Choosing, using and evaluating assessments in high volume scenarios
Case Study – High volume assessment process redesign
Questions and discussion
Slide 2
2. 2
Group Talent at RBS
Operating model
Slide 3
RBS Group Talent
Director,
Group Talent
People Strategy,
Insights
& Change
Early Career Resourcing
Executive
Search
Diversity
and Inclusion
Organisational
Effectiveness
Culture
Development
Learning
3. 3
Resourcing Operating Model
Retail
M&IB
Business Services
Group Functions
Regional
Fulfilment–ServiceandOperations
Fulfilment–Specialist
Fulfilment–Executive
Support–Assessment
Support–CandidateAttraction
Support–Policyandgovernance,BusinessMgtetc
Slide 5
Fulfilment–EarlyCareer
Support–CandidateSourcing
High Volume Assessment
Choosing, using and evaluating tools
Slide 6
4. 4
Sifting vs. Selection
Selecting out (Sifting)
Early removal of those who are least suited to
the role
– Often online, high volume, low touch
assessments
Selecting in
More detailed assessment to identify the best
candidates
– Offline, higher touch assessments
Sifting options in high volume contexts
Realistic job previews
Fact based disqualifiers (e.g. experience, qualifications etc)
Aptitude/ability tests (verbal, numerical etc)
Personality/behaviour/motivation tools
Telephone screening interviews
Situational judgment measures
5. 5
Sifting options in high volume contexts
Realistic job previews
– Pros
• Simple, easy to implement within an Applicant Tracking system
• Associated with positive candidate experience
– Cons
• Require regular updates and reviews to ensure that they remain current
• Efficacy cannot be guaranteed as they encourage self selection rather than judge a candidate’s suitability
Fact based disqualifiers (e.g. experience, qualifications etc)
– Pros
• Simple, easy to implement within an Applicant Tracking system
– Cons
• Blunt tools, easy to deselect people on spurious essential qualifications, insufficient on their own to manage the size of an
applicant pool
Aptitude/ability tests (verbal, numerical etc)
– Pros
• Generally solid predictors of performance in roles where problem solving and/or capacity to learn is important
– Cons
• Potential adverse impact by gender or ethnicity, so should not be the sole decision-making tool
Personality/behaviour/motivation tools
– Pros
• Relevance for tapping into corporate culture/values
• Capacity to differentiate across roles (e.g. sales vs service orientation)
• Generally limited adverse impact
– Cons
• Fakeability (therefore generally less predictive than tests)
Telephone screening interviews
– Pros
• Can be structured and standardised – good way of managing volume prior to a f2f assessment
• Good candidate experience – opportunity to speak to a real person
– Cons
• More resource intensive than other sift tools
• Cannot be used in very high volumes
Situational judgment measures
– Pros
• Provide good insight into the nature of the role
• Generally measure different attributes to tests/questionnaires
– Cons
• Complexity to design and maintain in a fast changing environment.
Define the role and identify the critical capabilities and experience
Estimate the overall candidate : hire ratio
- previous experience
- industry benchmarking
Choose assessments and their position in the funnel (estimate the selection
ratios necessary at each stage)
- Assessment characteristics, best practice, adverse impact etc
Model the end to end process, integrate assessments, candidate comms etc
Pilot
Live
Evaluation
Staging the process
6. 6
Factors to consider in the choice of Assessments
Quality/Predictiveness
– www.psychtesting.org.uk
Cost
Level of detail
Time taken (both candidate and assessor)
Relevance (tailored vs. off-the-shelf)
Level of touch
Technological sophistication
Candidate experience
Evaluation standards
Time to hire
Cost per hire
Candidate experience feedback
Quality of hire
– Typically composite metrics including:
– Hiring manager satisfaction survey data
e.g. would you rehire this individual?, rate your overall satisfaction with quality of hire
– New hire appraisal data
e.g. average performance rating for new employees in the first 12 months, percentage
of employees gaining “achieves expectations or above” in first 12 months
– Employee retention % in first 12 months of employment
– Good QoH measurement is elusive
7. 7
Service and Operations
End-to-end recruitment process re-engineering case study
Slide 13
Drivers for Change
Business structure/strategy
– Restructures in 2009 set out the need for a more cost-effective
process
– Greater use of online sifting
– Greater use of more resource intensive assessments
Short term tenure
– Internal research highlighted that lack of clarity about job
requirements was a key reason for STT
Candidate feedback
– Desire to enhance candidate experience
Slide 14
8. 8
Service and Operations Roles
Circa one third of total hires into the UK business
Roles which make up much of the bank’s headcount
– Annual salary range c. £14k-£20k
Currently up to 50,000 applicants per year
Found across the bank including Retail Banking and Business Services
Five main role clusters:
– Telephony Sales
– Telephony Customer Service
– Face to Face Customer Service
– Payments and Processing
– Customer Administration
Automated recruitment process using online methodology supported by
i-GRasp (ATS)
Slide 15
Process Workflow
• Realistic Job Preview
– Candidates preview job cluster scenarios to allow informed self selection
• Online Assessment (WSA)
– Structured questions to assess motivational, attitudinal and preference attributes for each volume role
• Telephone assessment
– Includes numeracy questions
• Line Manager Interview
– Structured Interview and Work sample assessmentsto assess observed task capabilities (e.g., customer
engagement, learning procedures, carrying out administrationtasks)
Enhanced Process
*Non Contact roles
Telephone
Online
Face-
to-face
**Contact roles
Key: *Non Contact Roles: Cash &
Coin
Pass / Fail
Planning &
Attraction
Job
Search
Realistic
Job
Preview
Group
Screen
Capture
Candidate
Details
(CV upload)
Diversity
Capture
Candidate
Offer
Pre-
Employment
Screening /
Onboarding
90%
Progression
Rate
Offer &
Onboarding
Resourcing Services / Consultant / Manager Advice & Overview
Candidate
Decline
Online
Assessment
50%
Progression
Rate
Telephone
Assessment
55%
Progression
Rate
Line
Manager
Interview
50%
Progression
Rate
**Contact Roles:F2F Customer Service; F2F Customer Adviser; Web Customer Adviser; Telephone Customer Service;
Telephone Customer Adviser; Collections & Recoveries; Complaints; Processing / Customer AdministrationBusiness & Resourcing
Business
Resourcing
Workforce /
Demand
Planning
Vacancy
Authorisation
RFI / Position
Creation
Vacancy
Keying
Attraction
9. 9
Assessments Implemented
Realistic Job Preview
– Information on typical activities undertaken and the characteristics of people who are likely to enjoy
the role and those who are unlikely to enjoy the role
– Encourages more self-selection by candidates by providing more detailed and realistic understanding
of the role
Online Assessment (WSA)
– Motivational, attitudinal and preference attributes for each of the role clusters
– Improved upon an older, “one size fits all” personality assessment
– Content designed in-house and delivered by a third party test engine
– And integrated into the ATS in order to provide a seamless assessment process
Telephone Assessment
– Delivered by an HR Shared Services team set up specifically to support recruitment activities
– Focuses on key predictors of success, such as capability to engage with customers
– Includes simple numerical screening questions
– On-screen interview script for recruiters
Line Manager Interview
– Short term tenure assessment
– Interview questions (situational and competency based)
– Work sample exercises
Slide 17
Slide 18
Telephone Interview
12. 12
Evaluation – External benchmarking
Online assessment the lowest cost of benchmarked organisations – c. 10%
of the cost of the most expensive solution
– In-house online assessment per-candidate cost of c. £0.25
Time savings and opportunity cost
– 30% reduction on duration of telephone interview compared to pre-intervention
– Overall costs of telephone assessment amongst the lowest of comparable
companies surveyed
Candidate quality improvements
– LMI selection ratios moved from 1 in 3 to better than 1 in 2 (reduced drop-out and
better candidates)
Process improvement
– Increase in candidate self-selection at the start of the process
– Reduction in candidate drop-out later on in the process
Candidate Experience
– Candidate feedback is the most positive across all RBS recruitment processes
(Alexander Mann, 2011)
Slide 23
Evaluation – QoH
Qualitative feedback from hiring managers in 2011 that candidate QoH has improved.
Improvement in quality of candidates in comparison with 2009 performance evaluations.
Key differences:
2009
% Scoring very
highly (8-10)**
2011
% Scoring very
highly (8-10)**
Improvement
Has a positive attitude to their
work
52% 61% 17%
Engages well with customers 54% 63% 17%
Comes into work on time 55% 76% 38%
Is good at generating
leads/making sales
37% 52% 40%
Slide 24
13. 13
Future Plans
Review process in line with culture change/Purpose Vision and Values
initiative
Continued focus on enhanced candidate experience
– Multi-media online assessment simulations
– Explore SJT options
Integration with RBS Apprentice programmes
Extend workflow methodology to additional roles
– Branch managers, Assistant Branch managers
– Improved global consistency – e.g. RBS Citizens in USA
A few last points
COMMON MISCONCEPTIONS ABOUT ASSESSMENT
Assessments do not automatically improve the quality of the overall applicant
pool
– they provide a more accurate stack-rank of your applicant pool.
New assessment processes do not automatically improve hire quality
– unless the current process is missing out the best people or you are attracting
more people
IN SUMMARY, ASSESSMENTS BRING THE MOST VALUE…
When there are a lot of applicants for a few roles (i.e. you can be highly
selective)
When high performers bring about much greater business benefit than low
performers
When the current success rate is low