1. Evaluations of Bullet Lead Evidence: Are Mock Jurors as Smart as They Think They Are?Evaluations of Bullet Lead Evidence: Are Mock Jurors as Smart as They Think They Are?
Suzanne O. Kaasa, Erin K. Morris, Tiamoyo Peterson, & William C. Thompson
University of California, Irvine
0
10
20
30
40
50
60
70
80
90
100
Strong Worthless Unknown Control
ExperimentalConditions
BACKGROUNDBACKGROUND
•Since the 1960s, bullet lead evidence has been used by the FBI in court
proceedings.
•Recently, the National Research Council (2004) issued a report questioning the
usefulness of this evidence for linking a crime scene bullet to a specific box of
bullets owned by the defendant (its diagnosticity).
•This year, the FBI has announced that it will no longer be performing analysis of
bullet lead evidence.
•Research on juror comprehension and usage of bullet lead evidence is still
important for two reasons: 1) previous cases that used this type of evidence may
come under increased scrutiny, and 2) the statistical reasoning used by jurors to
evaluate bullet lead evidence is similar to that used with other forms of forensic
evidence.
•This study was conducted in order to evaluate juror understanding of bullet lead
evidence and assess the relationship between individual differences and guilt
verdicts.
PROCEDUREPROCEDURE
•295 college students formed mock juries ranging from 4-6 people.
•Jurors were presented with a summary of a murder trial in which a key issue was
whether the fatal bullet came from a box of bullets owned by the defendant (it was
a “match”). The strength of the bullet lead evidence (its diagnosticity) was varied
by condition.
•Strong Condition: the likelihood of a match was far higher if the murder bullet
came from the defendant’s box than if it came from another source.
•Weak Condition: the likelihood of a match was approximately the same if the
murder bullet came from the defendant’s box as if it came from another source.
•Unknown Condition: jurors were told that a match was found, but were not
given specific statistical data about the likelihood of a match. This condition is
most like real world testimony about bullet lead.
•Control Condition: no bullet lead evidence was presented.
GENERAL FINDINGSGENERAL FINDINGS
Jurors in the Strong Condition were significantly more likely to find the defendant
guilty than those in the Worthless and Control Conditions. Jurors in the Unknown
Condition were more likely to convict than those in the Control Condition (see
Figure 1).
EFFECTS OF CONFIDENCEEFFECTS OF CONFIDENCE
Jurors were asked to rate how confident they were in their ability to draw correct conclusions from
numerical data. Two groups were formed based on a median split: Confident and Non-confident (see
Figure 2).
•Confident vs. Non-confident:
•Non-confident jurors found the defendant guilty at equally low rates for all conditions.
•Confident jurors did vary significantly in their guilt verdicts by condition.
•Confident jurors found the defendant to be guilty at significantly higher rates from Non-confident
jurors in the Strong and Unknown Conditions.
•Confident Jurors Across Conditions:
•Confident jurors in the Strong Condition found the defendant guilty at significantly higher rates than
those in the Worthless and Control Conditions.
•Confident jurors in the Unknown Condition found the defendant guilty at significantly higher rates
than those in the Worthless Condition.
CONFIDENCE AND PERFORMANCECONFIDENCE AND PERFORMANCE
•Amount of mathematical course experience showed a weak positive correlation with confidence,
but was not associated with guilt verdicts.
•Confident jurors did not show a clear advantage over Non-confident jurors in their recall of the
given statistics.
•Confident jurors were more likely to perceive the importance of the statistical information given
about the bullet lead evidence.
•Confident jurors gave weight to bullet lead evidence in the Unknown Condition, even though
necessary statistical information was not presented.
References and Further ReadingReferences and Further Reading
•National Research Council.. Forensic analysis: Weighing bullet lead evidence (2004)
•William C. Thompson, Analyzing the Relevance and Admissibility of Bullet Lead Evidence: Did the NRC Report Miss the Target?
Jurimetrics (in press)
•Edward J. Imwinkelried & William A. Tobin, Comparative Bullet Lead Analysis (CBLA) Evidence: Valid Inference or Ipse Dixit? 28
Okla.City L.R. 43 (2003)
•Michael O. Finkelstein & Bruce Levin, Compositional Analysis of Bullet Lead as Forensic Evidence, 13 Brooklyn J. Law & Policy 119, n.1
(2005)
DISCUSSION: ARE JURORS AS SMART AS THEY THINK THEYDISCUSSION: ARE JURORS AS SMART AS THEY THINK THEY
ARE?ARE?
Overall, mock jurors were able to distinguish between strong and worthless bullet lead evidence
when presented with relevant statistics, and gave more weight to evidence that was highly diagnostic
of guilt. However, these results appear to be driven primarily by confident mock jurors.
Those who were not confident in their ability to draw correct conclusions from numerical data
tended to perform the same across conditions; that is, they found the defendant to be guilty at
equally low rates regardless of the evidence presented. The data indicate that this is not due to a
general memory deficit (i.e., Non-confident jurors remembered the given statistics as well as
Confident jurors), but instead may be due to a lack of understanding of the importance of the various
statistics. Non-confident jurors’ self-assessment of mathematical ability appears to be an accurate
reflection of their performance, given that they do not perceive the importance of key statistics and
their guilt judgments are not affected by the strength of the evidence presented.
Confident jurors, on the other hand, may be overestimating their abilities to form correct conclusions
from numerical data. Although they are more likely to rate key statistics as important, and to
correctly use statistics when they are presented with them, these jurors are also more likely to give
weight to evidence without known diagnosticity. That is, Confident jurors in the Unknown
Condition assume diagnosticity even though key statistics are missing.
Further research may explore the extent to which these findings are applicable to other types of
forensic evidence that require the integration of multiple pieces of statistical evidence and/or the
ability to recognize when critical statistics are absent. Moreover, researchers may explore potential
methods to facilitate juror understanding and usage of statistical evidence.
Figure 1: Percentage of Jurors
Voting Guilty by Condition
Figure 2: Percentage of Confident and
Non-confident Jurors Voting Guilty
0
10
20
30
40
50
60
70
80
90
100
Strong Worthless Unknown Control
ExperimentalConditions
Confident Non-confident