1. Chapter 3:
Personality Psychology as Science:
Research Methods
The Personality Puzzle
Sixth Edition
by David C. Funder
Slides created by:
Tera D. Letzring
Idaho State University
© 2013 W. W. Norton & Company, Inc.
1

2. Objectives
• Discuss research methods that are particularly
important to personality psychology
• Discuss the difference between scientific
education and technical training
• Discuss aspects of data quality
• Discuss common research designs
• Discuss some statistical issues
• Discuss research ethics
© 2013 W. W. Norton & Company, Inc.
2

3. Psychology’s Emphasis
on Methods
• Psychologists sometimes seem to know more
about research methods than about the mind
and behavior.
• Goal: improving hypotheses
• Question everything, be skeptical, think
analytically.
© 2013 W. W. Norton & Company, Inc.
3

4. Scientific Education and
Technical Training
• Technical training
• Scientific education
– Question what is already “known”
– Learning to explore the unknown
– Research: the exploration of the unknown
© 2013 W. W. Norton & Company, Inc.
4

5. Quality of Data: Reliability
• Definition
• Measurement error
– Also called error variance
– The cumulative effect of extraneous influences
• States versus traits
© 2013 W. W. Norton & Company, Inc.
5

6. Quality of Data: Factors that
Undermine Reliability
•
•
•
•
Low precision of measurement
The state of the participant
The state of the experimenter
The environment
© 2013 W. W. Norton & Company, Inc.
6

7. Quality of Data:
Enhancing Reliability
• Be careful
• Use a standardized procedure or protocol
• Measure something that is important and
engages participants
© 2013 W. W. Norton & Company, Inc.
7

8. Quality of Data: Enhancing Reliability
• Aggregation
– Allow random influences to cancel each other out
– Especially important for predicting behavior
– Spearman-Brown formula
© 2013 W. W. Norton & Company, Inc.
8

9. Quality of Data: Validity
• Definition
• A “slippery” concept
– Reliability is necessary but not sufficient for
validity.
– Invokes the idea of “ultimate truth”
© 2013 W. W. Norton & Company, Inc.
9

10. Quality of Data: Validity
• Difficulty of measuring a construct
– Assessing personality is similar to testing a theory.
• Construct validation
– Gather as many measurements as possible.
– Look for the ones that hang together.
© 2013 W. W. Norton & Company, Inc.
10

11. Quality of Data: Generalizability
• The distinction between reliability and validity
is regarded as “fuzzy” by some.
• Definition
© 2013 W. W. Norton & Company, Inc.
11

12. Quality of Data: Generalizability
• Generalizability over participants
– Gender bias: Women are more likely to volunteer
and show up.
– Shows versus no-shows
– Cohort effects: the tendency of a group of people
living at a particular time to be different in some
way from those who lived earlier or later
– Ethnic and cultural diversity
© 2013 W. W. Norton & Company, Inc.
12

13. Quality of Data: Generalizability
• The burden of proof
– Avoid simplistic generalizations to members of
other cultures and people in different times
(including differences).
– Those who question the generalizability of a study
should propose when, how, and why it is not
generalizable.
© 2013 W. W. Norton & Company, Inc.
13

14. Research Design: Case Method
• Definition
• Can yield explanations of particular events,
general lessons, and scientific principles
• Case studies of ourselves
© 2013 W. W. Norton & Company, Inc.
14

15. Research Design: Case Method
• Advantages
– Describes the whole phenomenon
– Source for ideas
– Sometimes necessary for understanding an
individual
• Disadvantages
– No control
– Findings must be confirmed by other cases, which
is not usually possible.
15
© 2013 W. W. Norton & Company, Inc.

16. Research Design:
Experimental Method
• Definition: a research technique that
establishes the causal relationship between an
independent variable (x) and a dependent
variable (y) by randomly assigning participants
to experimental groups characterized by
differing levels of x, and measuring the
average behavior y that results in each group
© 2013 W. W. Norton & Company, Inc.
16

17. Research Design:
Experimental Method
• Test differences between groups with
statistical tests to determine if the difference
is larger than would be expected by chance
© 2013 W. W. Norton & Company, Inc.
17

18. Research Design:
Experimental Method
Leaders (high
power)
Leaders’
assistants (low
power)
© 2013 W. W. Norton & Company, Inc.
Rank list of
items needed to
survive in a
lifeboat on the
open sea
Measure
interpersonal
sensitivity
18

19. Research Design:
Experimental Method
F (1, 72) = 4.91, p = .03
© 2013 W. W. Norton & Company, Inc.
19

20. Research Design:
Correlational Method
• Definition
• Scatter plot
• Correlation coefficient
© 2013 W. W. Norton & Company, Inc.
20

21. Research Design:
Correlational Study
Measure power
Measure
interpersonal
sensitivity
Determine
the
relationship
r = .25
© 2013 W. W. Norton & Company, Inc.
21

22. Research Design: Comparing the
Experimental and Correlational
Methods
• Both attempt to assess the relationship
between two variables.
• The statistics (with two groups) are
interchangeable.
• The experimental method manipulates the
presumed causal variable, and the
correlational method measures it.
© 2013 W. W. Norton & Company, Inc.
22

23. Research Design: Comparing the
Experimental and Correlational
Methods
• Only experiments can assess causality.
– Correlational studies: unknown direction of cause;
third-variable problem
© 2013 W. W. Norton & Company, Inc.
23

24. Research Design: Comparing the
Experimental and Correlational
Methods
• Complications with experiments
– Uncertainty about what was really manipulated
• Third-variable problem
– Can create unlikely or impossible levels of a
variable
– Often require deception
– Not always possible
• Experiments are not always better.
© 2013 W. W. Norton & Company, Inc.
24

25. Research Design:
Representative Design
• Frequent concern: representativeness of
participants
• Less frequent, but important, concerns
– Representativeness across stimuli
– Representativeness across responses
© 2013 W. W. Norton & Company, Inc.
25

26. Research Design:
Representative Design
• Solution: use a representative design
– Seldom done because it is expensive and timeconsuming
© 2013 W. W. Norton & Company, Inc.
26

27. Thinking About Representativeness
• How is the psychology of today’s college
students different from that of their parents?
Would the conclusions of research done with
college students apply to their parents? What
areas are most likely to be different?
© 2013 W. W. Norton & Company, Inc.
27

28. Thinking About Representativeness
• Is research done with the predominantly
white college students in Western cultures
also relevant to members of ethnic minorities
or to people who live in other cultures? In
what areas would you expect to find the most
differences?
© 2013 W. W. Norton & Company, Inc.
28

29. Significance Testing
• Statistical significance: a result that would
only occur by chance less than 5% of the time
• p-level: probability level of obtaining a result
from a statistical test if there really is no
difference between groups or no relationship
between variables
• Null-hypothesis significance testing (NHST)
© 2013 W. W. Norton & Company, Inc.
29

30. Significance Testing:
Problems with NHST
• The logic is difficult to describe (and
understand).
• “Significant” does not necessarily mean strong
or important.
• The criterion for significance is an arbitrary
rule of thumb.
• Chances of significance vary with sample size.
© 2013 W. W. Norton & Company, Inc.
30

31. Significance Testing:
Problems with NHST
• Nonsignificant results are often interpreted as
“no result.”
• Only provides information about the
probability of one type of error
– Type I error vs. Type II error
• Cannot really tell you if a result is important
© 2013 W. W. Norton & Company, Inc.
31

32. Correlations and Effect Sizes
• Effect size definition
• More meaningful than a significance (p) level
• Correlation coefficient
– Can be used for correlational and experimental
studies
• Between -1 and +1
– 0 = no relationship
• Positive and negative correlations
© 2013 W. W. Norton & Company, Inc.
32

33. Correlations and Effect Sizes
• Use for prediction
• Interpreting correlations
– Look at the actual size
– r2 = percent of variance explained; “a terrible way
to evaluate effect size” (p. 92)
– Binomial Effect Size Display (BESD)
© 2013 W. W. Norton & Company, Inc.
33

34. BESD
r = .00
Drug
No drug
Lived
50
50
100
© 2013 W. W. Norton & Company, Inc.
Died
50
50
100
100
100
200
34

35. BESD
r = .30
Drug
No drug
Lived
50 + (r*100)/2
= 65
50 - (r*100)/2
= 35
100
© 2013 W. W. Norton & Company, Inc.
Died
50 - (r*100)/2
= 35
50 + (r*100)/2
= 65
100
100
100
200
35

36. BESD
r = .25
High I.S.
Low I.S.
62.5
37.5
100
37.5
62.5
100
High
power
Low
power
100
100
200
If you are worried about being interpersonally
sensitive, do you want to have high power?
© 2013 W. W. Norton & Company, Inc.
36

37. Thinking About Statistical Issues
• Let’s say we find that you score 4 points
higher on a “conscientiousness” test than
does another person. Alternatively, imagine
that women score 4 points higher on the same
test, on average, than men do. In either case,
is this difference important? What else would
we have to know to be able to answer this
question?
© 2013 W. W. Norton & Company, Inc.
37

38. Research Ethics
• The uses of psychological research
– Make sure it is not harmful, or at least that the
potential harm does not outweigh the potential
good.
• Truthfulness
– Avoid plagiarism and fabrication of data.
© 2013 W. W. Norton & Company, Inc.
38

39. Research Ethics: Deception
•
•
•
•
Definition
Purpose: usually to make the research realistic
APA guidelines
Review by the Institutional Review Board (IRB)
or Human Subjects Committee (HSC)
© 2013 W. W. Norton & Company, Inc.
39

40. Research Ethics: Deception
• Arguments in favor of deception
– Informed consent
– It usually does no harm.
– Certain topics cannot be investigated without
deception.
© 2013 W. W. Norton & Company, Inc.
40

41. Research Ethics: Deception
• Arguments against deception
– Informed consent for deception is not possible.
– When does the deception stop?
– Harms credibility of psychology
– Alternative: Investigate topics in the real world.
• What do you think about deception? Is it
justified?
© 2013 W. W. Norton & Company, Inc.
41

42. Clicker Question #1
In order to say that one variable caused
another, a researcher must
a)calculate the correlation between the
variables.
b) conduct an experiment.
c) construct a BESD.
d) use deception.
© 2013 W. W. Norton & Company, Inc.
42

43. Clicker Question #2
In order for data to have a high degree of
validity
a)they must also have a high degree of
reliability.
b) they must come from an experiment.
c) they must have low generalizability.
d)one must know the ultimate truth about the
construct being assessed.
© 2013 W. W. Norton & Company, Inc.
43

44. Clicker Question #3
The case method should be used when
a)the researcher is especially concerned that the
results have high generalizability.
b)the researcher wants to establish the cause of
a particular behavior.
c)the researcher wants to have lots of control.
d)there is an individual that the researcher
wants to understand as fully as possible.
© 2013 W. W. Norton & Company, Inc.
44