Ian Newcombe from RBS's presentation at our May 2013 Building Capability Conference. 'The role of integrated online assessment in high volume recruitment'

1. 1
The role of integrated online assessment in high volume
recruitment
Ian Newcombe C.Psychol, C.Sci, AFBPsS
Head of Assessment
Ian.Newcombe@rbs.co.uk
Objectives
 Group Talent at RBS
 Choosing, using and evaluating assessments in high volume scenarios
 Case Study – High volume assessment process redesign
 Questions and discussion
Slide 2

2. 2
Group Talent at RBS
Operating model
Slide 3
RBS Group Talent
Director,
Group Talent
People Strategy,
Insights
& Change
Early Career Resourcing
Executive
Search
Diversity
and Inclusion
Organisational
Effectiveness
Culture
Development
Learning

3. 3
Resourcing Operating Model
Retail
M&IB
Business Services
Group Functions
Regional
Fulfilment–ServiceandOperations
Fulfilment–Specialist
Fulfilment–Executive
Support–Assessment
Support–CandidateAttraction
Support–Policyandgovernance,BusinessMgtetc
Slide 5
Fulfilment–EarlyCareer
Support–CandidateSourcing
High Volume Assessment
Choosing, using and evaluating tools
Slide 6

4. 4
Sifting vs. Selection
 Selecting out (Sifting)
Early removal of those who are least suited to
the role
– Often online, high volume, low touch
assessments
 Selecting in
More detailed assessment to identify the best
candidates
– Offline, higher touch assessments
Sifting options in high volume contexts
 Realistic job previews
 Fact based disqualifiers (e.g. experience, qualifications etc)
 Aptitude/ability tests (verbal, numerical etc)
 Personality/behaviour/motivation tools
 Telephone screening interviews
 Situational judgment measures

5. 5
Sifting options in high volume contexts
 Realistic job previews
– Pros
• Simple, easy to implement within an Applicant Tracking system
• Associated with positive candidate experience
– Cons
• Require regular updates and reviews to ensure that they remain current
• Efficacy cannot be guaranteed as they encourage self selection rather than judge a candidate’s suitability
 Fact based disqualifiers (e.g. experience, qualifications etc)
– Pros
• Simple, easy to implement within an Applicant Tracking system
– Cons
• Blunt tools, easy to deselect people on spurious essential qualifications, insufficient on their own to manage the size of an
applicant pool
 Aptitude/ability tests (verbal, numerical etc)
– Pros
• Generally solid predictors of performance in roles where problem solving and/or capacity to learn is important
– Cons
• Potential adverse impact by gender or ethnicity, so should not be the sole decision-making tool
 Personality/behaviour/motivation tools
– Pros
• Relevance for tapping into corporate culture/values
• Capacity to differentiate across roles (e.g. sales vs service orientation)
• Generally limited adverse impact
– Cons
• Fakeability (therefore generally less predictive than tests)
 Telephone screening interviews
– Pros
• Can be structured and standardised – good way of managing volume prior to a f2f assessment
• Good candidate experience – opportunity to speak to a real person
– Cons
• More resource intensive than other sift tools
• Cannot be used in very high volumes
 Situational judgment measures
– Pros
• Provide good insight into the nature of the role
• Generally measure different attributes to tests/questionnaires
– Cons
• Complexity to design and maintain in a fast changing environment.
Define the role and identify the critical capabilities and experience
Estimate the overall candidate : hire ratio
- previous experience
- industry benchmarking
Choose assessments and their position in the funnel (estimate the selection
ratios necessary at each stage)
- Assessment characteristics, best practice, adverse impact etc
Model the end to end process, integrate assessments, candidate comms etc
Pilot
Live
Evaluation
Staging the process

6. 6
Factors to consider in the choice of Assessments
 Quality/Predictiveness
– www.psychtesting.org.uk
 Cost
 Level of detail
 Time taken (both candidate and assessor)
 Relevance (tailored vs. off-the-shelf)
 Level of touch
 Technological sophistication
 Candidate experience
Evaluation standards
 Time to hire
 Cost per hire
 Candidate experience feedback
 Quality of hire
– Typically composite metrics including:
– Hiring manager satisfaction survey data
e.g. would you rehire this individual?, rate your overall satisfaction with quality of hire
– New hire appraisal data
e.g. average performance rating for new employees in the first 12 months, percentage
of employees gaining “achieves expectations or above” in first 12 months
– Employee retention % in first 12 months of employment
– Good QoH measurement is elusive

7. 7
Service and Operations
End-to-end recruitment process re-engineering case study
Slide 13
Drivers for Change
 Business structure/strategy
– Restructures in 2009 set out the need for a more cost-effective
process
– Greater use of online sifting
– Greater use of more resource intensive assessments
 Short term tenure
– Internal research highlighted that lack of clarity about job
requirements was a key reason for STT
 Candidate feedback
– Desire to enhance candidate experience
Slide 14

8. 8
Service and Operations Roles
 Circa one third of total hires into the UK business
 Roles which make up much of the bank’s headcount
– Annual salary range c. £14k-£20k
 Currently up to 50,000 applicants per year
 Found across the bank including Retail Banking and Business Services
 Five main role clusters:
– Telephony Sales
– Telephony Customer Service
– Face to Face Customer Service
– Payments and Processing
– Customer Administration
 Automated recruitment process using online methodology supported by
i-GRasp (ATS)
Slide 15
Process Workflow
• Realistic Job Preview
– Candidates preview job cluster scenarios to allow informed self selection
• Online Assessment (WSA)
– Structured questions to assess motivational, attitudinal and preference attributes for each volume role
• Telephone assessment
– Includes numeracy questions
• Line Manager Interview
– Structured Interview and Work sample assessmentsto assess observed task capabilities (e.g., customer
engagement, learning procedures, carrying out administrationtasks)
Enhanced Process
*Non Contact roles
Telephone
Online
Face-
to-face
**Contact roles
Key: *Non Contact Roles: Cash &
Coin
Pass / Fail
Planning &
Attraction
Job
Search
Realistic
Job
Preview
Group
Screen
Capture
Candidate
Details
(CV upload)
Diversity
Capture
Candidate
Offer
Pre-
Employment
Screening /
Onboarding
90%
Progression
Rate
Offer &
Onboarding
Resourcing Services / Consultant / Manager Advice & Overview
Candidate
Decline
Online
Assessment
50%
Progression
Rate
Telephone
Assessment
55%
Progression
Rate
Line
Manager
Interview
50%
Progression
Rate
**Contact Roles:F2F Customer Service; F2F Customer Adviser; Web Customer Adviser; Telephone Customer Service;
Telephone Customer Adviser; Collections & Recoveries; Complaints; Processing / Customer AdministrationBusiness & Resourcing
Business
Resourcing
Workforce /
Demand
Planning
Vacancy
Authorisation
RFI / Position
Creation
Vacancy
Keying
Attraction

9. 9
Assessments Implemented
 Realistic Job Preview
– Information on typical activities undertaken and the characteristics of people who are likely to enjoy
the role and those who are unlikely to enjoy the role
– Encourages more self-selection by candidates by providing more detailed and realistic understanding
of the role
 Online Assessment (WSA)
– Motivational, attitudinal and preference attributes for each of the role clusters
– Improved upon an older, “one size fits all” personality assessment
– Content designed in-house and delivered by a third party test engine
– And integrated into the ATS in order to provide a seamless assessment process
 Telephone Assessment
– Delivered by an HR Shared Services team set up specifically to support recruitment activities
– Focuses on key predictors of success, such as capability to engage with customers
– Includes simple numerical screening questions
– On-screen interview script for recruiters
 Line Manager Interview
– Short term tenure assessment
– Interview questions (situational and competency based)
– Work sample exercises
Slide 17
Slide 18
Telephone Interview

10. 10
Slide 19
Telephone Interview (Numeracy)
LMI STT Risk Assessment

11. 11
LMI Situational Questions
LMI Work Sample Test
Slide 22

12. 12
Evaluation – External benchmarking
 Online assessment the lowest cost of benchmarked organisations – c. 10%
of the cost of the most expensive solution
– In-house online assessment per-candidate cost of c. £0.25
 Time savings and opportunity cost
– 30% reduction on duration of telephone interview compared to pre-intervention
– Overall costs of telephone assessment amongst the lowest of comparable
companies surveyed
 Candidate quality improvements
– LMI selection ratios moved from 1 in 3 to better than 1 in 2 (reduced drop-out and
better candidates)
 Process improvement
– Increase in candidate self-selection at the start of the process
– Reduction in candidate drop-out later on in the process
 Candidate Experience
– Candidate feedback is the most positive across all RBS recruitment processes
(Alexander Mann, 2011)
Slide 23
Evaluation – QoH
 Qualitative feedback from hiring managers in 2011 that candidate QoH has improved.
 Improvement in quality of candidates in comparison with 2009 performance evaluations.
Key differences:
2009
% Scoring very
highly (8-10)**
2011
% Scoring very
highly (8-10)**
Improvement
Has a positive attitude to their
work
52% 61% 17%
Engages well with customers 54% 63% 17%
Comes into work on time 55% 76% 38%
Is good at generating
leads/making sales
37% 52% 40%
Slide 24

13. 13
Future Plans
 Review process in line with culture change/Purpose Vision and Values
initiative
 Continued focus on enhanced candidate experience
– Multi-media online assessment simulations
– Explore SJT options
 Integration with RBS Apprentice programmes
 Extend workflow methodology to additional roles
– Branch managers, Assistant Branch managers
– Improved global consistency – e.g. RBS Citizens in USA
A few last points
COMMON MISCONCEPTIONS ABOUT ASSESSMENT
 Assessments do not automatically improve the quality of the overall applicant
pool
– they provide a more accurate stack-rank of your applicant pool.
 New assessment processes do not automatically improve hire quality
– unless the current process is missing out the best people or you are attracting
more people
IN SUMMARY, ASSESSMENTS BRING THE MOST VALUE…
 When there are a lot of applicants for a few roles (i.e. you can be highly
selective)
 When high performers bring about much greater business benefit than low
performers
 When the current success rate is low

14. 14
Questions
Slide 27