Psychometrics Slides

Psychometrics 101
Emma Rush

Session for the Legal Education and Training
Group

Emma Rush Consulting February 2011

Agenda
Gain a better understanding of the ‘added value’ a
psychometric can bring within a legal environment
Get a clear overview of the six main instruments on
the market and when they can be used
Understand the resource issues of psychometrics:
the cost, the time required, the trained personnel
needed
Develop your own case, if required, for introducing
psychometrics to all or any of the ‘people processes’

Emma Rush Consulting

Starting with me
As a consultant:
Hogan™, MBTI Step II® and OPQ32r™ for coaching and
career consulting
Previous law firm:
16PF® for Assessment for Development Centre
MBTI Steps I & II ® for team workshops
FIRO ® for Leadership programme
FSI:
16PF ® mapped on to a competency framework for partner
recruitment and partnership track
FIRO® for Assertiveness
Destiny for secretarial recruitment

Brief history of psychometrics
Began in late C19th with “intelligence” testing
Francis Galton, UK: Anthropometrics – taking a
mathematical approach to the measure of individual
differences – eugenics
Charles Spearman, USA 1910 – contributed ‘g’ or
the measurement of general intelligence factor
And “factor analysis” – the means by which
psychometric tests are found to be valid



1917 Stanford-Binet test: first standardised
test of intelligence
1949: First publication of the 16PF by
Raymond Cattell (Spearman’s student)
1962: MBTI first published – type-based
instrument; four preference scales



1963 First appearance of the “Big Five” of
personality factors
1970: Cronbach (Cronbach’s Alpha):
emphasises the importance of validity – being
able to predict accurately what a person will
do


1984: OPQ32 introduced, published by SHL
R&D by Peter Saville and Roger Holdsworth
1986: Hogan’s Personality Inventory first
published
2004 Saville and Holdsworth ousted from
SHL’s board
2008 Saville’s Personality Questionnaire /
Wave introduced


Key concepts
Reliability: The degree to which an
instrument measures the same way each time it
is used under the same conditions with the
same subjects – its “consistency”
Validity: The extent to which you actually
are measuring what you are professing to
measure
Type vs. Trait: Type gives you a preference,
trait measures how you compare to a wider
population

Where to use psychometrics

Used in thousands of organisations at all
stages of the employee “life cycle”

We’re looking at three: Selection;
Development ; Promotion


USING PSYCHOMETRICS IN
SELECTION


Why select with psychometrics?
Type of intervention Validity ( r ) where 1.0 is
perfect prediction
General Mental Ability Tests .51
Structured Employment Interviews .51
Used together .63
Job Knowledge Tests .48
Unstructured Employment Interviews .38
Assessment Centres .37
Reference checks .26
Job experience (years) .18
Education (years) / Interests .10
Graphology .02
Chance 0


Why select with psychometrics?


Why select (or train) with
psychometrics?
To make the business case you first need
to know:
What it currently costs (directly and indirectly) to
recruit, train and retain someone so you can
measure/prove an improvement
Exactly what you are looking for – job criteria,
competency framework
What outcomes you require from training
interventions


In addition testing…
Allows for better and clearer discrimination between
candidates – “beyond the good egg factor”
Supports your diversity initiatives: more transparent,
fairer and no adverse impact
Allows a better fit with your job criteria or
competency framework
Administrative convenience (depending on your
resources)
Costs and development time are reasonable


What do you test?

Five key areas to measure How do they get measured?
Knowledge Areas on which traditional or
unstructured approaches
Experience concentrate – psychometrics can
still assess and assist

Behaviours Areas where online tests and
structured approaches can assess
Personality these areas and increase validity
Capability of recruitment
In descending order of ease to
Motivation change – motivation hardest


Intellectual Ability

Job-based or reasoning tests – used
frequently at trainee level, but not at higher
and more expensive level
Standard intellectual ability tests:
Verbal reasoning
Numerical reasoning
Thinking style


PERSONALITY INSTRUMENTS IN
USE


A word of warning to begin!

“..it seems that users of some
questionnaires become attached to the
tests that “look right” or appear to be
appropriate, which is known as faith
validity..” (Saville, 1975)
For every argument there is a counter-
argument


The Big , in in test terms

OPQ32r®
The 16PF5 ®
NEO PI-R™
Hogan Personality Inventory and
Development Survey™
Saville Wave Professional Styles®


How valid are they?
Figures taken from Saville’s report How valid is your
questionnaire?

High
validity

Moderate
validity

Chance

A time comparison
Instrument No of Questions Typical Completion time
OPQ32i 416 60 minutes
NEO-PI-R 240 40 minutes
Wave Professional 216 40 minutes
Styles
Hogan 206 30 minutes
16PF5 185 30 minutes
Saville’s Personality 72 13 minutes
Questionnaire


Tests to avoid

Ipsative Tests
Tests without appropriate norms
Tests with low reliability estimates
Old tests that have not been re-evaluated in
the last 10 years.
Glossy, packaged tests with no psychometric
details


And not to be used for selection

DISC: “The test suffers from questionable
reliability and unknown validity”

MBTI: a “type” based instrument so
without norm group validity


Costs

Using psychometrics may have a long
term benefit but also has a short term cost
Per instrument – from £12 to £298 per report
Time: an hour’s minimum feedback
Online – no additional cost to set up
Reputational risk


Which tests to use?
OPQ32r™
Pros: an “Occupational” questionnaire so about the
“you at work”
“r” is the latest version, and has three rather than
four item choices
Published by SHL and designed by Professor Peter
Saville
Can be mapped onto a competency framework
Multiple report options
Relatively quick to complete

OPQ32r™

Relationships with Feelings and
People Emotion
Influence Emotion
Sociability Dynamism
Empathy
Thinking Style
Analysis
Creativity and Change
Structure


OPQ32r™
Cons:
32 factors – do they add anything extra? They take
longer to give feedback on
Overlap between factors e.g. relationships with people /
feelings and emotion
Now quite old (1984), doesn’t cover to same degree
motivations, values, influencing style, leadership
Is it an ideal fit with professional services – norm groups
used?
Hard to get an overview of scores in the reports, more
interpretative than detailed

16PF®
Pros: A combination of five global factors and
sixteen primary factors – gives some granularity but
not overwhelming
Used world-wide for recruitment and development
Can be mapped onto a competency framework
Factors relate to universal traits rather than forced
into three pronged model
Reports give practitioner option as well as
interpretative reports
Relatively quick to complete

16PF®

Cons: Meta-perceptions: candidates frequently
report “I do this differently in social situations”
Measures drive (obliquely) but not specifically
motivations
Reasoning questions can’t be relied on as
measure of intellectual ability, particularly for
lawyers
Is it an ideal fit for professional services – norm
groups used?


Hogan Personality Inventory/Development
Survey/Motivations, Preferences and Values Inventory
(all™)

Pros: Uniquely covers both “bright” and
“dark” side of personality
Covers what drives individuals, what derails
them and what they value in an organisation
Particularly suited for senior recruits e.g..
lateral hires, as reports based on strengths
and competencies for leaders


Hogan Personality Inventory/Development
Survey/Motivations, Preferences and Values Inventory
(all™)

Cons: quite American – language in reports,
language in questions
Three separate questionnaires to complete
Relevance to professional services?
Relevance to more junior roles in professional
services?
Supplied in UK by smaller distributors e.g.
Mentis; Psychological Consultancy


NEO
The NEO PI-R was developed by Costa
and McCrae
Five domains measured: Neuroticism,
Extraversion, Openness to Experience,
Agreeableness and Conscientiousness.


NEO

Each five domains is divided into six
facets: thirty areas of personality which
predict employees’ work behaviour
including leadership style, team style,
decision-making and stress management.
240-item questionnaire either online or by
paper and pencil
The current UK Edition was published in
2004.

Saville Consulting’s Wave ®
Professional Styles edition
To quote Peter Saville: “Saville Consulting
Wave® Professional Styles measures motives,
talents, preferred culture and competency
potential in one dynamic online questionnaire”
A wealth of detail: 4 Clusters; 12 Sections; 36
Dimensions; 108 Facets
More reliable
Quick to complete
Normed for professionals

DEVELOPMENT AND PERFORMANCE


Ongoing development: workshops
and coaching
Specific learning e.g. leadership,
assertiveness
Coaching
Team workshops


Why train with psychometrics?

The sheer cost of training
Have to do something 200 times to be
completely proficient in a new skill -
accelerates the process of self-awareness
Assessing motivation means you can see in
advance:
How this person likes to work
What motivates this person to work
What types of goals this person likes

What type of intervention?

How you like to learn e.g. Honey and
Mumford’s Learning Styles Questionnaire
What motivates you – how do we reinforce
learning e.g. Hogan MPVI
Why you need to learn e.g. FIRO-B with
leadership
How you operate in a team e.g. MBTI Step II
What the real issues are in coaching e.g.
Saville Consulting Wave® Professional Styles

Team Interventions

Three instances you may instigate a team
workshops:
Forming a team
Dealing with team performance
Team building


FIRO-B
A unique way of looking at how a team
operates
Three “scales”, no typology, scored from 1 to
9
Inclusion; Control; Affection (or Openness)
An “Expressed” score: what you show to
others
A “Wanted” score: what you want from
others but may not show

Team Workshops

MBTI Steps I and II
Uniquely lends itself to group feedback with a
reported type followed by self-assessed type
and ending with best fit
Step II adds factors to the four dimensions
giving more granularity and better comparison
between team members
Can be a powerful aide to raising self-
awareness


However…

Can feel over-simple even at Step II
MBTI Step I gives only four dimensions for
comparison – does this adequately explain how
a team interacts or personality works?
Slip between “reported type” and “best fit” can
be as much as 25% e.g.. only 75% of people
confirm their MBTI results
Lacks real scientific validity


Belbin
Dr Meredith Belbin; dates from the 1970s
Belbin’s Nine Team Roles
Plant Co-ordinator

Monitor-Evaluator Resource-
Investigator

Implementer Completer-finisher

Team worker Shaper

Specialist

PROMOTION


A form of selection

The same instrument should be used for
promotion purposes as for selection for
reasons of:
Consistency
Fairness
Transparency
Proper metrics of progress (from selection to
now)


How do you make the best
decision on promotion?

The more structured methods you use the
better validity you gain
Assessment centres alone are less valid than
when combined with e.g.
Structured interviews
Psychometrics


Questions?


Suppliers
OPP:http://www.opp.eu.com/Pages/home.asp
x
SHL:http://www.shl.com/default.aspx
Hogrefe: http://www.hogrefe.co.uk/
Get Feedback: http://www.getfeedback.net/
Saville Consulting:
http://www.savilleconsulting.com/contact/conta
ct.aspx

Organisations

The Psychometrics Forum
http://www.psychometricsforum.org/


Training

I recommend working out which test(s) you want
to use, and then qualifying with that test
publisher
For example OPP’s Level B course qualifying
you in both MBTI® and 16PF®
Or Saville Consulting qualifying you in Wave™


Psychometrics Slides

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Psychometrics Slides

Ähnlich wie Psychometrics Slides (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Psychometrics Slides