[Workshop] Implementation of screening (Oct10)

Alex Mitchell www.psycho-oncology.info/workshop
Department of Cancer & Molecular Medicine, Leicester Royal Infirmary
Department of Liaison Psychiatry, Leicester General Hospital
IPOS 2010IPOS 2010
WORKSHOP Day 2
Implementation of Screening:
Screening studies, Short methods, HADS and longer methods,
implementation, future of screening
WORKSHOP Day 2
Implementation of Screening:
Screening studies, Short methods, HADS and longer methods,
implementation, future of screening

Schedule Day 2Schedule Day 2
930-10.00 – Introduction to research task 1. design 2. evaluation
10.00-11.00 – T3 Screening in Cancer: Instruments & Validity
Break
11.30 – 12.30 – Group work #2
Lunch
1.30-2.30 – T4 Screening in Cancer: Implementation and future
Break
3.00 – 4.00 – Presentation of Research task

Group Work #2Group Work #2
930-10.00 – Introduction, groups and issues
10.00-11.00 – T1 Basic science of screening
Break
11.30 – 12.30 – Group task #1
Lunch
1.30-2.30 – T2 Symptoms, Burden, Help, Needs in Cancer
Break
3.00 – 4.00 – Evaluation of a screening paper

Group Work #2Group Work #2
Read paper in your group……..
1.What is being tested?
2.What is the comparison?
3.Is the tool effective?
4.Is the tool acceptable?
5.Did the tool make a difference?

T1. Are We Looking for Distress?T1. Are We Looking for Distress?
How Often
What method?

n=226
Comment: Frequency of cancer specialists
enquiry about depression/distress from
Mitchell et al (2008)

1,2 or 3 Simple
QQ
15%
Clinical Skills
Alone
73%
ICD10/DSMIV
0%
Short QQ
3%
Other/Uncertain
9% Other/Uncertain
2%
Use a QQ
15%
ICD10/DSMIV
13%
Clinical Skills
Alone
55%
1,2 or 3 Simple
QQ
15%
Cancer Staff
Current Method (n=226)
Psychiatrists
Comment: Current preferred method of eliciting
symptoms of distress/depression

1,2 or 3 Simple
QQ
24%
Clinical Skills
Alone
20%
ICD10/DSMIV
24%
Short QQ
24%
Long QQ
8%
Algorithm
26%
Short QQ
23%
ICD10/DSMIV
0%
Clinical Skills
Alone
17%
1,2 or 3 Simple
QQ
34%
Cancer Staff
Ideal Method (n=226)
Psychiatrists
Effective?
Comment: “Ideal” method of eliciting
symptoms of distress/depression according
to clinician

T2. Are We finding it?T2. Are We finding it?
How successful are we (routinely)?

Comment: Slide illustrates diagnostic
accuracy according to score on DT
11.8
15.4
30.4 28.9
41.9 42.9 40.7
57.1
82.4
66.7
71.4
15.8
25.0
26.1
24.4
19.4 19.0
33.3
21.4
11.8
22.2 14.3
72.4
59.6
43.5
46.7
38.7 38.1
25.9
21.4
5.9
11.1
14.3
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Zero One Two Three Four Five Six Seven Eight Nine Ten
Judgement = Non-distressed
Judgement = Unclear
Judgement = Distressed

0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Pre-test Probability
Post-testProbability
CHEMO+
CHEMO-
Baseline Probability
COMMU+
COMMU-
Detection sensitivity = 50.6%
Detection specificity = 79.4%
Overall accuracy = 65.4%.
Comment: Slide illustrates performance of chemotherapy vs community nurses in oncology T125 – Sat am

0
10
20
30
40
50
60
70
80
90
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
GP Accuracy – Detection of Distress by GHQ ScoreGP Accuracy – Detection of Distress by GHQ Score
McCall et al (2007) Primary Care Psychiatry - Recognition by Severity
Comment: Slide illustrates raw number
of people identified by severity on the
GHQ. Although the % detection
increases with severity, the absolute
number decreased due to falling
prevalence

0
0.05
0.1
0.15
0.2
0.25
0.3
Eight
N
ine
Ten
Eleven
Tw
elve
Thirteen
Fourteen
Fifteen
Sixteen
Seventeen
Eighteen
N
ineteen
Tw
entyTw
enty-one
Proportion Missed
Proportion Recognized
HADS-D

Testing Clinicians: A Meta-AnalysisTesting Clinicians: A Meta-Analysis
Methods (currently unpublished)
12 studies reported in 7 publications.
2 studies examined detection of anxiety,
8 broadly defined depression (includes HADS-T)
3 strictly defined depression and 7 broadly defined distress.
9 studies involved medical staff and 2 studies nursing staff.
Gold standard tools including GHQ60, GHQ12 HADS-T, HADS-D,
Zung and SCID.
The total sample size was 4786 (median 171).

Testing Clinicians: A Meta-AnalysisTesting Clinicians: A Meta-Analysis
All cancer professionals
SE =39.5% and SP =77.3%.
Oncologists
SE =38.1% and SP = 78.6%; a fraction correct of 65.4%.
By comparison nurses
SE = 73% and SP = 55.4%; FC = of 60.0%.
When attempting to detect anxiety oncologists managed
SE = 35.7%, SP = 89.0%, FC 81.3%.
Presented at IPOS2009

GPs vs Oncologists vs NursesGPs vs Oncologists vs Nurses
Who is better?
Bayesian analysis

0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
GP+
GP-
Nurse+
Nurse-
Oncologist+
Oncologists-
Comment: Doctors appear to be more
successful at ruling-in or giving a
diagnosis, nurses more successful at
ruling out

0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Ave Confidence+
Ave Confidence-
Above Ave Confidence+
Above Ave Confidence-
High Confidence+
High Confidence-
Low confidence = more cautious, fewer false positives, more false negatives
High confidence = less cautious, more false positives, low false negatives
p180

T3. Screening Tools in CancerT3. Screening Tools in Cancer
Clinician Opinion
Patient Opinion

Observation
Interview
Visual
Self-Report
Depression
Screening
DISCS
VA-SES
ET/DT
HAMD-D
17
PhysicalGeneral
Signs of
DS
6
CDSS#10
MADRAS
10
Trained
Confident
Skilled
Clinician
Alone
YALE
SMILEY

Clinicians Methods to Evaluate Depression
Unassisted Clinician Conventional Scales
Ultra-Short (<5) Short (5-10) Long (10+)Untrained Trained
Routine Implementation
Acceptability ?
Accuracy? Accuracy?
vs Comment: schematic overview of
methods to evaluate depression
example

Clinicians Methods to Evaluate Depression
Conventional Scales
Short (5-10) Long (10+)
HADS-D BDI
example example

Comment: This is a reminder of the
structure of the HADS scale, this version
adapter for cancer.

HADS – Pros vs ConsHADS – Pros vs Cons
ADVANTAGES DISADVANTAGES

HADS – Pros vs ConsHADS – Pros vs Cons
ADVANTAGES
Well known
Short (7 items)
Well tested
Depression & anxiety covered
Self-report
DISADVANTAGES
Can be too long
Validation stats not good
Which version?
Distress, anger, needs not
covered
Scoring complex
HADS-t not recommended
Royalty fee

Inadequate Data
(n=11)
No data (n= 250)
No reference standard
(n= 293)
Accuracy or Validity Analyses
(n= 210)
HADS Validity Analyses
(n=50)
HADS in Cancer
Initial Search (n= 768)
Scale
Types
Sample Size
(cases)
HADS-T
(n=26)
HADS-D
(n=14)
HADS-A
(n=10)
Less than 30
(n=22)
More than 100
(n=8)
30 to 100
(n=20)
Review articles (n= 16)
Depression
(n=22)
Any Mental Ill Health
(n=24)
Anxiety
(n=4)
Outcome
Measure
No interview standard
(n=149)

Validity of HADS vs depression (DSMIV)Validity of HADS vs depression (DSMIV)
SE 71.6% (68.3)
SP 82.6% (85.7)
Prev 13%
PPV 38%
NPV 95%

0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
HADS+
HADS-
HADS7v8+
HADS7v8-
Depression_HADS-d (7v8)

British Journal of Cancer (2007) 96, 868 – 874

0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
Distress
Thermometer
Anxiety
Thermometer
Depression
Thermometer
Anger
Thermometer
Ten
Nine
Eight
Seven
Six
Five
Four
Three
Two
One
Zero
Comment: Slide illustrates scores on ET
tool

ET - Table of Cut-PointsET - Table of Cut-Points
Distress
Thermometer
Anxiety
thermometer
Depression
Thermometer
Anger
Thermometer
Help
Thermometer Cut-point
Insignificant 39.0 25.6 50.1 55.7 54.3 0,1
Minimal 20.1 22.5 18.3 13.6 15.4 2,3
Mild 16.9 16.5 12.2 10.5 12.2 4,5
Moderate 12.0 14.5 9.8 6.6 6.6 6,7
Severe 11.9 20.8 9.5 13.6 11.2 8,9,10
p130

8%
DT
37%
DepT
23%
AngT
18%
AnxT
47%
4%
7%
1%
1%
9%
3%
0%
2%
4%
15%
3%
2%
Nil
41%
Non-Nil
59%
DT
AnxT AngT
DepT

T4. How Valid Are the ToolsT4. How Valid Are the Tools

Validity of Methods to Evaluate Depression
Unassisted Clinician Conventional Scales
Ultra-Short (<5) Short (5-10) Long (10+)Untrained Trained

DT vs HADS-T Validity (n=660)DT vs HADS-T Validity (n=660)
SE SP AUC CUT
DT – 71.9% 78.4% 0.814 cut point >=4
AnxT – 75.7% 73.4% 0.821 cut point >=5
DepT – 77.6% 82.2% 0.855 cut point >=3
AngT – 77.5% 77.6% 0.823 cut point >=2
HelpT - 69.1% 80.8% 0.809 cut point >=3

0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
HADSd+
HADSd-
HADS-T+
HADS-T-
HADS-A+
HASD-A-
Depression_HADS

0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
1Q+
1Q-
DT+
DT-
2Q+
2Q-
HADSd+
HADSd-
HADS-T+
HADS-T-
BDI+
BDI-
EPDS+
EPDS-
HADS-A+
HASD-A-
Depression_all

0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
DT+ [N=4]
DT+ [N=4]
1Q+ [N=4]
1Q- [N=4]
2Q+
2Q-
DT/IT+
DT/IT-
HADST+ [N=13]
HADST+ [N=13]
PDI+
PDI-
Mitchell AJ. Short Screening Tools for Cancer Related Distress A Review and Diagnostic Validity Meta-analysis JNCI (2010) in press
Distress

Validity of DT vs depression (DSMIV)Validity of DT vs depression (DSMIV)
SE 80%
SP 60%
PPV 32%
NPV 93%

DT vs DSMIV DepressionDT vs DSMIV Depression
SE SP PPV NPV
DTma 80.9% 60.2% 32.8% 92.9%
DTLeicesterBW 82.4% 68.6% 28.0% 98.3%
DTLeicesterBSA 100% 59.6% 26.8% 100%
BSA = British South Asian
BW= British White

T5. How to Choose A Cut-OffT5. How to Choose A Cut-Off

Distress ThermometerDistress Thermometer

Distress Thermometer – Pooled TableDistress Thermometer – Pooled Table
Score
Ransom
2006
Tuinman
2008
Mitchell
2009
Lord
2010
Hoffman
2004
Gessler
2009
Clover
2009
Jacobsen
2005 Sum
Proporti
on
Zero 68 38 61 123 14 27 65 71 467 18.4%
One 72 31 42 68 5 26 39 46 329 12.9%
Two 77 22 35 44 5 18 30 54 285 11.2%
Three 65 37 42 46 8 23 45 46 312 12.3%
Four 51 29 29 30 8 7 21 31 206 8.1%
Five 41 46 62 40 11 13 41 48 302 11.9%
Six 38 32 23 28 2 16 26 31 196 7.7%
Seven 36 21 23 38 2 15 32 16 183 7.2%
Eight 18 12 18 29 6 9 19 15 126 5.0%
Nine 16 5 8 14 3 3 13 9 71 2.8%
Ten 9 4 7 20 4 0 9 13 66 2.6%
Sum 491 277 350 480 68 157 340 380 2543
Proportion 19.3% 10.9% 13.8% 18.9% 2.7% 6.2% 13.4% 14.9%

Distress Thermometer – Pooled
Proportion
18 .4 %
12 .9 %
11.2 %
12 .3 %
8 .1%
11.9 %
5.0 %
2 .8 % 2 .6 %
7.7%
7.2 %
0.0%
2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
16.0%
18.0%
20.0%
Zero One Two Three Four Five Six Seven Eight Nine Ten
Insignificant SevereModerateMildMinimal
p124
50%

SampleSample
We analysed data collected from Leicester Cancer Centre
from 2008-2010 involving 531 people approached by a
research nurse and two therapeutic radiographers.
We examined distress using the DT and daily function
using the question:
“How difficult have these problems made it for you to do
your work, take care of things at home, or get along
with other people?”
“Not difficult at all =0; Somewhat Difficult =1; Very
Difficult =2; and Extremely Difficult =3”

Dysfunction in 531 cancer patientsDysfunction in 531 cancer patients
55.7%
34.3%
7.3%
2.6%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
Unimpaired Mild Moderate Severe

Unimpaired by DT ScoreUnimpaired by DT Score
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
1 2 3 4 5 6 7 8 9 10 11

18%
DepT
23%
Distress
69%
Dysfunction
76%
0.3%
3% 2%
26%28% 22%
Of the 293 Non-Nil
Dysfunction
Distress
DepT

Mean DT Scores?Mean DT Scores?
Unimpaired Mild Moderate Severe
Mean DT Score 2.1 4.1 5.9 6.5
Std Deviation 2.54 3.0 2.56 3.59
Sample Size 296 182 39 14
Simplified DT Range* 0-3 4-5 6-7 8-10

DT distribution by ImpairmentDT distribution by Impairment
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0 1 2 3 4 5 6 7 8 9 10

Typically severely impared
Typically mod impared
Typically mildly impared
Typically unimpared
None at all

Extreme and incapacitating
Very Severe and very disabling
Moderately Severe and disabling
Moderate and quite disabling
Moderate and somewhat disabling
Mild-Moderate and slight disabling
Mild but not particularly disabling
Very mild and not disabling
Minimal but bearable
Minimal and not problematic
None at all

Dt vs DysfunctionDt vs Dysfunction
ROC plot from Book 1
0.00 0.25 0.50 0.75 1.00
0.00
0.25
0.50
0.75
1.00
Sensitivity
1-Specificity
Distress Thermometer(+ve), M(-ve)

Optimal Cut to Define Distress on DTOptimal Cut to Define Distress on DT
At a cut-off of 2v3 (>=3)
Sensitivity =67.8%; PPV =60.3%; UI+ = 0.409
Specificity = 68.9%; NPV = 70.3%; UI- = 0.484
Sensitivity =58.9%; PPV =65.6%; UI+ = 0.386
Sensitivity =50.9%; PPV =67.85; UI+ = 0.345

T6. Screening in Cancer: ImplementationT6. Screening in Cancer: Implementation
Clinician Opinion
Patient Opinion

Comment: Slide illustrates actual gain in
meta-analysis of screening
implementation in primary care

Screen
Routine vs At-Risk vs Identified
Low High
Follow-up Care
?? Desire for Help
Meetable Unmet Needs

800 Patients Approached
100 Not Willing (13%) 700 Patients Willing (87%)
500 Staff Willing (71%)TAU
402 Data Collected (80%)Screen Data
Leicester: DT/ET ImplementationLeicester: DT/ET Implementation T177 t680

Pre-Post Screen - DistressPre-Post Screen - Distress
Before After
Sensitivity of 49.7% 55.8% =>+5%
Specificity of 79.3% 79.8% =>+1%
PPV was 67.3% 70.9% =>+4%
NPV was 64.1% 67.2% =>+3%
There was a non-significant trend for improve detection sensitivity (Chi² =
1.12 P = 0.29).

Qualitative Aspects: CommunicationQualitative Aspects: Communication
DISTRESS
43% of CNS reported the tool helped them talk with the patient
about psychosocial issues esp in those with distress
28% said it helped inform their clinical judgement
DEPRESSION
38% of occasions reported useful in improving communication.
28.6% useful for informing clinical judgement

2x2 Clinician Help Table : ACTUAL HELP2x2 Clinician Help Table : ACTUAL HELP
Clinician thinks:
Unmet Needs
Clinician thinks no
Unmet Needs
Patient Says:
Help Wanted
=> Intervention => Low grade
Patient Distressed => Intervention =>??
Patient Not
distressed or
Help Not Wanted
=> Monitor? => discharge?

2x2 Clinician Help Table : ACTUAL HELP2x2 Clinician Help Table : ACTUAL HELP
Clinician thinks:
Unmet Needs
Clinician thinks
no Unmet Needs
Patient Says:
Help Wanted (60)
Helped 21/35
(60%)
Helped 11/23
(48%)
Patient
Distressed
Helped 65/102
(63%)
Helped 31/62
(50%)
Patient Not
distressed or
Help Not Wanted
Helped 8/35
(23%)
Helped 20/117
(17%)

b. Intervention and helpb. Intervention and help
PREDICTORS
1. patient desire for help
2. number of unmet needs
3. clinicians confidence
4. patient reported anger
p179

RCT using DT Carlson et al 2010RCT using DT Carlson et al 2010
Screening for Distress in lung and breast cancer
outpatients: A randomized controlled trial Linda Carlson
Tom Baker Cancer Centre, University of Calgary
1) Minimal Screening: the Distress Thermometer (DT)
[n=365]
2) Full Screening: DT, Problem Checklist, Psychological
Screen for Cancer (PSSCAN) [n=391] a personalized
report
3) Triage: Full screening plus optional personalized phone
triage [378]

Advanced AspectsAdvanced Aspects
Algorithms
Structured interviews
Computerized testing
Item-banking
Screening in subgroups
p643
p454

Cancer Population
CNS Assessment
Possible case
Depression
Screen #1
+ve
n = 200
No Depression
Sp 55%
Se 70%
n = 800
N = 1000
TP = 140
FP = 360
Probable Non-Case
TN =440
FN = 60
PPV 28% NPV 88%
Screen #1
-ve
Yield TP = 140
TN = 440
FN = 60
FP = 360
NPV 88%
PPV 28%
Sp 55%
Se 70%

Cancer Population
CNS Assessment
Possible case
Depression
Screen #1
+ve
n = 200
No Depression
Sp 55%
Se 70%
n = 800
N = 1000
TP = 140
FP = 360
Probable Non-Case
TN =440
FN = 60
PPV 28%
Oncologist Assessment Sp 80%
Sp 40%
NPV 88%
Probable Depression
TP = 56
FP = 72
Probable Non-Case
TN =288
FN = 84
PPV 44% NPV 77%
Screen #1
-ve
Screen #2
+ve
Screen #2
+ve
Cumulative Yield TP = 56
TN = 728
FN = 144
FP = 72
NPV 83%
PPV 44%
Sp 91%
Se 28%

Credits & Acknowledgments
Elena Baker-Glenn University of Nottingham
Paul Symonds Leicester Royal Infirmary
Chris Coggan Leicester General Hospital
Burt Park University of Nottingham
Lorraine Granger Leicester Royal Infirmary
Mark Zimmerman Brown University, Rhode Island
Brett Thombs McGill University Canada
James Coyne University of Pennsylvania
Nadia Husain University of Leicester
For more information www.psycho-oncology.info

FURTHER READING:
Screening for Depression in Clinical Practice An
Evidence-Based guide
ISBN 0195380193
Paperback, 416 pages
Nov 2009
Price: £39.99

[Workshop] Implementation of screening (Oct10)

Recommended

Recommended

More Related Content

Similar to [Workshop] Implementation of screening (Oct10)

Similar to [Workshop] Implementation of screening (Oct10) (20)

More from Alex J Mitchell

More from Alex J Mitchell (20)

Recently uploaded

Recently uploaded (20)

[Workshop] Implementation of screening (Oct10)