Behavioural event interviewing science or religion

Behavioural Event Interviewing:
Science or Religion

NZPsS Annual
Conference
Rotorua - July 2010

Keith McGregor
Personnel Psychology NZ Ltd
www.ppnz.com
21/07/2010 1
PERSONNEL PSYCHOLOGY NZ LTD

21/07/2010 2

Structured Interviews
1. Base questions on a person spec from a job analysis
2. Ask exactly the same questions of each candidate
3. Limit prompting, follow-up questioning and elaboration
4. Use better types of questions, namely;
- future situational questions (What would you do if ...?)
- past experience questions (Tell us about a time ..?)
- work history questions (What experience with ...?)
- job knowledge, competency questions (How do you?)

Campion, M., A, D. Palmer, K, et al. (1997). "A Review of Structure
in the Selection Interview." Personnel Psychology 50: 655-702.

21/07/2010 3


5. Use longer interviews or more questions
6. Withhold access to ancillary information
(application forms, resumes, test scores, etc.)
7. Don’t allow questions from candidates until
after the interview
8. Rate each answer using multiple scales
9. Use detailed anchored rating scales
10. Take detailed notes

21/07/2010 4


11. Use multiple interviewers on panels
12. Use the same interviewers across all
candidates
13. Don’t discuss candidates or answers between
interviews
14. Provide extensive interview training
15. Use statistical methods to analyse the scores

21/07/2010 5

Hypothetical Situational Questions

• Identify required competencies
• Obtain examples of critical incidents
• Create a scoring template
• Score the answers
• Appoint the person with the best score
• “What would you do if you discovered a co-
worker was planning to leak information?”

21/07/2010 6

Behavioural Event Questions

• Identify required competencies
• Write questions focusing on past behaviours
• Situation – Behaviour – Outcome - Motivation
• Create a scoring schedule
• Score the answers
• Appoint the person with the best score
• “Can you tell us about a time when….”

21/07/2010 7

Question Sent To Candidate

Customer Service Orientation:
Tell us about a time when you recognised that
more was required than the customer (internal
or external) asked for and what you then went
on to do.
Tell us of a situation when you have had to say
‘no’ to a customer or client, what did you do?

(Taken from a 1996 questionnaire sent to a candidate prior to interview.)

21/07/2010 8

Inconsistent Application
70% based their questions on the person specification
14% used ad-lib questions.
43% described using a set format
70% included at least some generic questions, such as
describing strengths and weaknesses, that were not
specifically job related.
28% had complete freedom in the questioning process
22% who stayed with the set questions. The balance
used a semi-structured approach.
52% indicated they used some form of rating.
Taylor, P., Y. Keelty, et al. (2002). "Evolving Personnel Selection
Practices in New Zealand Organisations and Recruitment Firms."
New Zealand Journal of Psychology 31(1): 8-18.
21/07/2010 9

Reasons for Variation
• Interviewers may be unaware of the research
• needs for power or autonomy may be frustrated by the
structured approach
• Interview may become a boring, monotonous exercise
• Applicants may be alienated by the formality of the
process or may find the process frustrating
• Process may be at odds with the values and beliefs of
the organisation
• There may be financial or time constraints
• Applicants perceive both the organisation and interview
more positively when unstructured or semi-structured.
van der Zee, K. I., A. B. Bakker, et al. (2002). "Why are structured
interviews so rarely used in personnel selection?" Journal of Applied
Psychology. 87(1): 176-184.
21/07/2010 10

Coaching
Formal coaching and preparation for public service workers
scheduled to attend a structured interview
- Improvement in interview performance
- More favourable impression with interviewers
- Correlated with performance on the job
A de facto IQ test - candidates demonstrating ability to
learn and apply the taught skills

Maurer, T. J., J. M. Solamon, et al. (2001). "Interviewee Coaching,
Preparation Strategies, and Response Strategies in Relation to
Performance in Situational Employment Interviews: An extension of
Maurer, Solamon, and Troxtel (1998)." Journal of Applied Psychology
86(4): 709-717.
Maurer, T. J. and J. M. Solamon (2006). "The Science And Practice Of A
Structured Employment Interview Coaching Program." Personnel
Psychology 59(2): 433-456.

21/07/2010 11

Impression Management
Assertive tactics - ingratiation, self-promotion
Defensive tactics - excuses, justifications, apologies
Applicants search environment for cues and tailor their self
presentation to the situation, e.g. cues inherent in the type
of question being asked
Experience based questions can evoke self promotion
tactics such as claiming responsibility for successes or
enhancing responsibilities
Situational questions may lead to ingratiation by supplying
answers likely to appeal to the interviewer
Ellis, A. P. J., B. J. West, et al. (2002). "The Use of Impression
Management Tactics in Structured Interviews: A Function of Question
Type?" Journal of Applied Psychology 87(6): 1200-1208.
21/07/2010 12

The ‘Russia House’ Dilemma
“Okay? We show Bluebird the shopping list and he turns
out to be their asset not ours. Have I considered that
possibility? Ned, day and night I have considered little
else. If Bluebird is, if the girl is, if Barley is, if all or any of
the players is less than strictly kosher, the shopping list
will shine a very bright light up the anal orifice of the
United States of America.
…It will show the Sovs what we don’t know and how we
don’t know it …
Cleverly analysed, the shopping list can show them the
gaps in our intelligence gathering machinery”

John Le Carre, ‘The Russia House’
21/07/2010 13

Weapon of Choice
“A clever psychopath can present such a well-
rounded picture of a perfect job candidate that
even seasoned interviewers"
Hare, R., D (1993). Without Conscience: The Disturbing World of the
Psychopaths Among Us. New York, The Guilford Press.

“Questions are a short course for the subject on
how to lie to us”
“We cannot trust information we have to solicit”
Avinoam Sapir (1988) Laboratory for Scientific Interrogation. Personal
Communication.

21/07/2010 14

Johari Window
SELF
Things we Things we
KNOW DO NOT KNOW
about ourselves about ourselves

Things others PUBLIC BLIND
KNOW
about us
ARENA SPOT

OTHERS
FACADE UNKNOWN
Things others
DO NOT KNOW
about us

21/07/2010 15

Interview Implications

Will not discuss
Will discuss because they
openly are unaware
e.g. gender
bias

Will not discuss Will not discuss
because they do because they
not want you to are unaware
know e.g. e.g. self
procrastination sabotage

21/07/2010 16

Values and Attitudes
Unstructured interviewing was more effective than
structured interviewing in identifying counterproductive
work behaviours.
Allowing freedom to explore aspects of the interviewee’s
experience and attitudes produced more valid
assessments of personality traits and produced more
information with fewer personality related questions than
did interviews which were highly structured.
The degree of structure rather than the type of questions
was the differentiating factor.
Blackman, M. C. and D. C. Funder (2002). "Effective Interview Practices
for Accurately Assessing Counterproductive Traits." International
Journal of Selection and Assessment 10(1&2): 109-116.

21/07/2010 17

What is Being Measured?
Conventionally structured interviews (focusing on
credentials and achievements, descriptions of experience
and self evaluative information such as likes, dislikes,
strengths and weaknesses) were assessing social skills
and temperament.
Behaviour structured interviews, (focusing on technical
knowledge and behaviour description) were assessing
general mental ability and work knowledge.

Moscoso, S. (2000). "Selection Interview: A Review of Validity Evidence,
Adverse Impact and Applicant Reactions." International Journal of
Selection and Assessment 8(4): 237-247.

21/07/2010 18

Counterintuitive Hypothesis
Is increased validity of structured interview due to higher
reliability?
How many unstructured interviews equate to validity of one
structured interview?
“Perhaps the best interpretation of the mixed results in this
study as that they indicate that there is evidence that is
countertheoretical to the prevailing beliefs about the
structured interview and that additional research should be
conducted to understand this issue more fully.” (Bugger!)

Schmidt, F.L., & Zimmerman, R.D. (2004). A Counterintuitive Hypothesis
About Employment Interview Validity and Some Supporting Evidence.
Journal of Applied Psychology, 89(3), 553-561

21/07/2010 19

Six Pieces of Paper
RELIABILITY Police officers need to be supportive,
dependable and to make themselves available to other
police officers. Please describe a situation where you had
to keep going without quitting. What was the situation and
the demands put upon you? What exactly did you do?
What did you do to keep yourself going? How did you
support others? What was the result?
“These initial results show that the six-page competency-
based application form designed for police officer
recruitment obtains quite similar results to other more
expensive, labour-intensive selection processes.”

Lamsdale, C., R. Wood, et al. (1999). "An Alternative to an Assessment
Centre on Six Pieces of Paper?" International Journal of Selection and
Assessment 7(3): 170-176.

21/07/2010 20

Structured Interview Problems

• Job analysis seldom evidence based and
seldom mention environmental factors
• Narrow scope of inquiry – may miss key
behaviours
• Trying to quantify a subjective judgement
• Unnatural and artificial
• Everyone is different
• Stressful (unethical?)

21/07/2010 21

Does the Past Predict The Future?

• Theory states that past behaviour is the best
predictor of future behaviour
• But - we seldom have access to past behaviour
• We only have access to current behaviour
• Body language is unreliable
• Only verbal behaviour is reliable
21/07/2010 22

Described
versus
Demonstrated
Behaviour
• Described behaviour - what they talk about
• Demonstrated behaviour - how they talk about
it
• Demonstrated behaviour is the most reliable
indicator

21/07/2010 23

What Is Demonstrated?
David, can you please give us an example of a situation
where you had to deal with an angry customer, explain what
you did and what outcome resulted.
That’s easy. When I was working as a salesperson at Gifts-
For-All a customer came in with this ridiculous complaint. I
mean it is amazing what some people expect. Anyway, I
calmed her down and apologised for what had happened. I
told her it was a manufacturing error, it wasn’t really but that
always sounds good. I replaced her purchase and gave her
a discount voucher and she was really happy. I thought it
was a good outcome and my boss said I had shown
excellent customer service skills.

21/07/2010 24

Structure is Essential

• Unstructured interviewing is NOT the solution
• Elicits irrelevant information
• Relies on ‘gut feel’
• Decisions difficult to justify – “his eyes were too
close together”
• The question is not ‘structured’ versus
‘unstructured’ but what kind of structure

21/07/2010 25

Purpose of an Assessment

1. To enable us to predict
how a candidate is
most likely to behave
on the job
2. To use our prediction
to guide the selection
decision

21/07/2010 26

Objective of an Interview

The primary objective of the interview is to obtain
sufficient valid, unbiased behavioural data to
enable an accurate prediction of how the person
will behave in the workplace.

21/07/2010 27

“She Interviewed Well….”

• We are not interested in how they ‘perform’ in
the interview
• We are not looking for anything - we are
looking to see what is there
• Must leave a legally robust ‘audit trail’ showing:
– what information was obtained,
– how it was analysed and
– how it related to potential on-the-job
behaviour

21/07/2010 28

Natural Skills

As a generalisation, most people are natural
interviewers
How would your partner / relative go as a central
city bus driver?
What questions would you ask at a party to see if
someone would fit into your work environment?

21/07/2010 29

We Predict By Matching…

With the known
requirements of the role

How the person
normally behaves

21/07/2010 30

The Key Elements

Job Candidate
Knowledge Knowledge
Skills Skills
Attitudes Attitudes

Predicted
on-the-job
performance

21/07/2010 31

KSA’s
(Bloom’s Taxonomy)

Knowledge (Cognitive) – the what
Skills (Psychomotor) – the how
Attitude (Affective) – the will

For every inch training technology gains,
selection technology loses a mile - don’t
waste time assessing things you can train

21/07/2010 32

‘Natural Selection’

Humans have a natural
tendency to select OUT
rather than select IN
We really want the candidate
to be good but when all the
lights go red it’s all over

21/07/2010 33

‘Incompetency’ Profile

The norm is to use a competency profile - what
we looking for. These are often generic and not
based on empirical evidence – people have
usually ‘discussed’ them into existence
Ideally, the competency profile should include an
‘Incompetency’ profile – behaviours shown by
people who have failed in the role.

21/07/2010 34

The ‘Pure’ Interview

We do not need to know what the job is in order to
do the interview.
A good interview will produce two main pieces of
information:
– what motivates and de-motivates the candidate
– what competencies they have
Armed with this information it should then be
possible to predict how the candidate will behave
in any given job

21/07/2010 35

The Theoretical Basis
Ivan Petrovich Pavlov Burrhus Frederic Skinner

Classical/Respondent Operant/Instrumental

21/07/2010 36

Reinforcing and Aversive Stimuli

We have two poles

Things we move Things we move

TOWARDS AWAY FROM

DETAIL?
PEOPLE?
CONFLICT?
STATUS?

21/07/2010 37

Assessing Personality

We are unable to read other people’s minds.
All we can do is observe their behaviour.
Personality can therefore be viewed as the:
• Long term (stable)
• Probability (predictable)
• of Specific (definable)
• Behaviour (observable)

21/07/2010 38

Assessing Personality

There are no absolutes in human behaviour. A
person may need the company of others most of
the time but still need time alone
To assess ‘personality’ we need to be able to
assign probabilities to the behaviours
We need to recognise that the interview is an
unnatural environment – the candidate has no
competition, they are being evaluated

21/07/2010 39

Sampling Behaviour

• Sampling is a cornerstone of science
• To assess personality we need an unbiased
sample of behaviours
• The sampling process is systematic not
selective
• Secondary school, for example, is simply a
starting point, it has no special significance
• Anything can be included – sports, part time
jobs, research projects, trips
21/07/2010 40

Interview Structure

POSITIVES NEGATIVES

SKILLS

POSITIVES NEGATIVES

SKILLS

POSITIVES NEGATIVES

SKILLS

21/07/2010 41

Getting Started

Start with very broad, open questions:
‘Just to get the ball rolling, tell us a bit about
yourself.’
- Do they seek clarification?
- What do they talk about?
- How long do they spend on each topic?
- How organised or orderly are they?
- Are there any ‘weirdies’?
- How long are their replies - long / medium / short?

21/07/2010 42

Interview Structure
Work through entire academic and work history:
• ‘Going back to the dim dark past, what were
some of the ‘highs’ and ‘lows’ of your last
couple of years at school?’
• ‘After school you worked at Grange Ltd. What
was that like?’
• ‘What prompted you to move on?’
• ‘While you were with Pastrycooks International,
what aspects gave you the most job
satisfaction?’

21/07/2010 43

Drawing Out The Values

‘Brilliant’ ...
‘Great’ ...
What was it ‘Good’ ...
like working ‘Okay’ ...
there? ‘Interesting’ ...
‘Awful’
‘Ummm’

21/07/2010 44

Probe the Adjectives

Everyone speaks a different language

Never assume you know how they interpret words
like ‘challenging’ or ‘awful’

‘When you say ‘awful’, what sort of things are you
thinking of?’

21/07/2010 45

Interpretable Information

• ‘Interpretable’ information reveals values and
attitudes
• The facts of the situation are irrelevant
• Consider the responses of two people who
attended the same Outward Bound course:
1. “Terrible, disorganised, lousy food”
2. “Amazing, I learned so much about myself”

21/07/2010 46

Laddering
Laddering Up (Why? Why? Why?)
What was that job like? “Great, I worked in a team”
As opposed to? “Working on my own”
Which do you prefer? “Being part of a team”
Because? “I know I can get help if I need it”
As opposed to? “Having to work things out for myself”
You prefer help because? “I struggle to make decisions”

Laddering Down (How? How? How?)
You mentioned helping others. How do you do that?
“I always seem to know when people need help”
Which you achieve by?
“I guess I am always looking around to see what is going
on”
21/07/2010 47

Fly on the Wall

Can you get a fly-on-the-wall picture of the
candidate in the workplace?
How did you find that role?
Oh, great
Great in what way?
I was the youngest rep in the history of the
company
And what did you love getting your teeth into?
Winning customers of the other reps, that was a
real buzz
21/07/2010 48

Fly (cringing) on the Wall

What about the frustrations, if something was
going to get up Dave’s nose what would it be?
Some of the other reps
Go on?
Oh, they used to play these stupid tricks, in the
end I just refused to talk to them
Anything else apart from that?
Some of the customers could be difficult, I think
basically they were just plain thick

21/07/2010 49

Never Sell Before Deciding

• Candidates are coached to try to get the
interviewer talking to gain cues about how to
respond.
• Give away nothing until the end of the interview.
• If they want to know more about the job, ask
them to give you their best guess.
• “Just so we don’t double up, how about you tell
us what you think the key points are and we can
fill in the gaps.”

21/07/2010 50

Notetaking

Write as close to word-for-word as possible
• Ensures accurate analysis
• Wards off subsequent challenges
• EEO fair
• Allows the candidate to relax
• Gives interviewer time to review
• Prevents interview ‘white-out’

21/07/2010 51

Assessing Skills and Abilities

• Drill down into the detail
• Use ‘closed’ or directive questions
• Get very specific information
• Ask ‘dumb’ questions
• If inexperienced solicit ‘best guess’

21/07/2010 52

The CV contains the Questions

Read carefully through the C.V. and note any
activities which are close to those you require.
Use those as the basis for skill and ability
questions.
“I see here you set up a chart of accounts. What
on earth is a chart of accounts, how do you set
those up?”

21/07/2010 53

‘Unconscious’ Non-Answers
Sam, could you please elaborate on your role and your job with TAP
Inc., specifically the organisational climate re-engineering project.
My role was as an internal consultant employed by TAP and the role
extended straight across the organisation. I had responsibility for the
non-government accounts. I worked with the training department itself.
The team was directly responsible for facilitating organisational change.
We were involved at the invitation of other departments. The intention
was that any unit undergoing change would call us in to help. In some
cases the bosses said we had to be involved.
We tried to be involved as early as possible and worked in a facilitation
role. The idea was for us to be involved as soon as people recognised
they needed help. We identified the problem, the real problem, and
suggested solutions and in some cases we were involved in the
implementation.
21/07/2010 54

What did Sam actually do?

Could you give us examples - what were you involved in the whole
time? And what were some of the smaller activities? How was
consistency achieved across the organisation?
Our aim was to achieve consistency across the organisation by
restructure and climate change. It was reorganised into customer
groups. We lost about 200 people across the whole company.
For the climate change we analysed the existing situation. We prepared
an analysis plan, we read up on similar programmes, we looked at the
language, we held focus groups and we carried out surveys. We used
a number of different methods to gather the information we needed.
There were 6 of us working on the project. I was the team leader for
the data management. We developed a picture of the existing climate
and from here we looked at where we wanted to be.

21/07/2010 55

‘Unconscious’ Non-Answers

Sam, could you please elaborate on your role and your job with TAP
Inc., specifically the organisational climate re-engineering project.
My role was as an internal consultant employed by TAP and the role
extended straight across the organisation. I had responsibility for the
non-government accounts. I worked with the training department itself.
The team was directly responsible for facilitating organisational change.
We were involved at the invitation of other departments. The intention
was that any unit undergoing change would call us in to help. In some
cases the bosses said we had to be involved.
We tried to be involved as early as possible and worked in a facilitation
role. The idea was for us to be involved as soon as people recognised
they needed help. We identified the problem, the real problem, and
suggested solutions and in some cases we were involved in the
implementation.

21/07/2010 56

What did Sam actually do?

Could you give us examples - what were you involved in the whole
time? And what were some of the smaller activities? How was
consistency achieved across the organisation?
Our aim was to achieve consistency across the organisation by
restructure and climate change. It was reorganised into customer
groups. We lost about 200 people across the whole company.
For the climate change we analysed the existing situation. We
prepared an analysis plan, we read up on similar programmes, we
looked at the language, we held focus groups and we carried out
surveys. We used a number of different methods to gather the
information we needed.
There were 6 of us working on the project. I was the team leader for
the data management. We developed a picture of the existing climate
and from here we looked at where we wanted to be.

21/07/2010 57

Analysing the Results

• Avoid making a decision straight after the
interview - allow time for the face to fade.
• Read the notes through at least three times.
• Write out or highlight all the positive and
negative comments.
• Sort the comments into natural groupings -
‘people’, ‘learning’, ‘status’, etc.
• Prepare a competency report and a prediction
of on-the-job behaviour.

21/07/2010 58

Q-Sort Positives and Negatives

Being made prefect Not being listened to by pupils
Teachers giving me praise Missing out on 1st XV selection
Winning the 400m prize Practical jokes by childish
sales reps
Being the youngest customer
service rep in history of Not being invited to Board
company dinner
Winning the top sales prize Manager trying to tell me how
to do my job
Winning customers off the
other reps Customers kept cancelling
orders
Working for an elite company
Having to explain things
several times to clients

21/07/2010 59

Interpreting the Results

• Work on the obvious - don’t read in things that
are not there.
• Be hypercritical - if they can’t suppress negative
traits in a 40 minute interview they won’t stand a
chance on the job.
• Focus on the behaviours demonstrated in the
interview - sarcasm, criticism, talking about
people, money, results, etc.

21/07/2010 60

Reporting the Results

Always prepare a written prediction of how the
person will behave in the workplace, even if it is
just for yourself. Dig it out six months later and
see how close you were.
If you are assessing against competencies use the
data you have collected as the basis for the
assessment.
Have someone else review your analysis to ensure
consistency
21/07/2010 61

Competency Report
Competency Assessment Evidence
Teambuilding Not achieved Referred to teams once in 60
minutes. Resented team
activities at school and in first
job

Technical Knowledge Achieved Gave in-depth answers, reads
widely and has advanced
knowledge

Awareness of Others Not Achieved On five occasions referred to
people as ‘half-wits’ and
idiots. Made six comments
about the trouble people had
put him to
21/07/2010 62

Summary Report
Positives Attention and support from others, feeling valued, being
listened to, pleasant working conditions, freedom to
make his own decisions, relaxed environment
Negatives Repetitive, boring work, managers, people of high rank,
education system, bureaucracy, being told what to do,
co-workers
General Initially impressed as open and responsive with good
Summary knowledge. Successfully completed three major
projects. As interview progressed started criticising
and blaming others. Highly judgemental and
confrontational
Predicted He will begin well and have some early wins but then
performance start ‘falling out’ with managers and co-workers. Will
resign and blame others for his problems

Recommend- Suitable for genuine, short term contract but not for
ation permanent employment.

21/07/2010 63

Light Reading

Blackman MC, Funder DC. (2002). Effective Interview
Practices for Accurately Assessing Counterproductive
Traits. International Journal of Selection and Assessment,
10, 109-116.
Campion MA, Palmer DK, Campion JE (1997) A Review of
Structure in the Selection Interview. Personnel Psychology,
50, 655-702
Thompson, D. W.: Managing People: Influencing
Behaviour. St Louis, Miss, 1978, C.V. Mosby. Pp 101-118

21/07/2010 64

Finally – Getting to the Truth

21/07/2010 65

About PPNZ
Personnel Psychology NZ Ltd provides a comprehensive range of industrial and
organisational psychology services.
• Specialist training (e.g. ‘The Psychology of Management’, ‘Advanced
Interview Skills’ and ‘Managing Organisational Change’)
• In-depth executive appointment screening
• Individual psychological assessment and counselling for executives
• Mentoring, coaching and supervision
• Organisational climate surveys and stress audits
• Psychometric test development ( e.g. Selector PA, Career Step, E Profiler)
• Safety management
The director, Keith McGregor, is an industrial psychologist based in Lower Hutt,
New Zealand. His background includes twelve years as an occupational
psychologist in the Royal New Zealand Air Force and twenty in private practice
working with a wide range of private and public sector organisations in New
Zealand and Australia.

21/07/2010 66

Behavioural event interviewing science or religion

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Behavioural event interviewing science or religion

Ähnlich wie Behavioural event interviewing science or religion (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Behavioural event interviewing science or religion