The document summarizes issues with past literature on psychosocial interventions for cancer patients. It finds that the literature: 1) does not provide a credible basis for recommending interventions to patients or advocating for insurance coverage due to poor methodological quality; and 2) on average, finds no benefits for patients receiving common interventions like support groups. It outlines endemic problems like lack of intent-to-treat analyses, biased reporting of results, and misuse of statistical analyses. It calls for improvements like preregistering trial designs, incorporating the CONSORT checklist for reporting, and conducting simpler, fair tests of interventions for distressed patients.

1. Psychosocial Interventions in
Cancer Care: The Evidence is
Worse Than it First Looks
James C. Coyne, Ph.D.
University of Pennsylvania
With Special Thanks to Drs. Steve Palmer and Joan Cook—and to
NCI, SBM, the Cancer SIG, and my fellow panelists.

2. We must not allow a shared commitment to
improving the wellbeing of cancer patients to be
exploited with exaggerated claims and poorly
conceived, poorly conducted, and poorly
reported clinical trials.

3. How the Literature Fails Us
• Does not provide a rationale basis for
favorably recommending interventions to
cancer patients.
• Does not provide a credible basis for
advocating for reimbursement by third
party payers.

4. Most Robust Finding:
No Benefit
• Average cancer patient receiving interventions
does not achieve a reduction in distress.
• Average cancer patient does not benefit from
participating in:
Group Cognitive Behavior Stress
Management
Supportive Expressive Group
Psychotherapy
Writing exercise (0-3).

5. The Literature is Worse Than
it First Looks

6. The Literature is Worse Than It
First Looks
• Cannot accept positive appraisals of a particular
study or the literature at face value.
• Endemic confirmatory bias.
• Myth that combinations of similarly flawed
studies can yield an informative contribution to
the literature: blend them together, you get
tainted scrapple, not pate.

7. The Literature is Worse Than It
First Looks
• Late
adoption of CONSORT leaves an
accumulated literature with serious
methodological shortcomings.
• Adoption of CONSORT tends to be associated
with 30% reduction in confirmatory bias.
• “Best Foot Forward” takes precedence over best
available evidence.

8. CONSORT
A list of requirements for uniform reporting
of clinical trials with the overall aim of
improving the reporting of Randomized
Controlled Trials, to facilitate their critical
appraisal, and to facilitate their inclusion in
systematic reviews.
Published in 1996, revised 2001

9. CONSORT Checklist
Not a Cure or even a Treatment for an Ailing
Literature.
Not a Diagnostic Instrument.
More of a Screening, a First Assessment of
Transparency of Reporting.
Most Effective When Used Preventively by
Authors, Rather than Later By Readers.

10. What are the Endemic
Problems in the Design,
Conduct, and Reporting of
Trials?

11. Poor Rationale for Trials
• Claims of high prevalence of distress are
not reflected in samples.
• Distress and depression do not predict
survival.
• Salivary cortisol not related to progression.
• Measures of immune function not related
to progression.
• Psychological interventions do not effect
immune function in cancer patients.

12. The Norm: Lack of
Intent to Treat Analyses
• Data from patients who do not complete trial or
all measurements are discarded.
• “As treated” analyses ignore informative missing
data.
• Intervention and control patients have different
reasons for not providing data and this
introduces bias in the available data.
• “As treated” data do not generalize back to
patients entering a trial.

13. Are We Done Yet? Check the Data
Again and See if We Have Got a
Finding to Report
• A priori power analysis the occasional exception
rather than the rule.
• Operative Rule: Peek and stop when results are
looking good.
• Must beware of modest sized trials claiming
strong effects--likely to be false positives.
• Must beware of studies with odd numbers of
patients accumulated without a power analysis.

14. Telling It Like It Ain’t: All the
Results That Fit
• Primary endpoint typically needs to be inferred, not
stated.
• Ignore negative results for presumed endpoints:
Emphasize any positive effect, ignore larger number of
null findings.
• Favor secondary and subgroup analyses and endpoints
developed post hoc over negative findings for presumed
analyses.
• Discuss negative findings as if positive in subsequent
publications.
• Accommodate existing literature “as is” rather than
qualifying interpretation with reference to methodological
shortcomings.

15. Just What Is Wrong With Post Hoc
Subgroup Analyses?
• High profile papers in the behavioral medicine
literature routinely emphasize subgroup analyses
when they are positive in the face of negative primary
analyses (Classen et al, 2001; Schneiderman et al.,
2004).
• In the broader clinical trials literature, this practice is
uniformly criticized as inappropriate (Yusuf et al.,
1991).
• Unplanned subgroup analyses frequently yield
spurious results (Assman et al., 2000; Senn & Harrel,
1979), and “only in exceptional circumstances should
they affect the conclusions drawn from the trial”
(Brooks et al., 2004, p 229).

16. Reporting a Major Intervention Trial
Focusing on Distress (Part 1)*
• First report published in Archives of General
Psychiatry.
• CES-D data collected but not reported.
• Request for information: ‘Null, so not reported’.
• POMS data: Post hoc analyses suggested
strategy of dropping last observation for some
patients, allowing marginally significant result.
Classen, Butler, Koopman, Miller, DiMiceli, Giese-Davis, Fobair,
Carlson, Kraemer & Spiegel 2001

17. Reporting a Major Intervention Trial
Focusing on Distress (Part 2)*
• Second report of same trial in Journal of Consulting and
Clinical Psychology.
• First paper cited in passing, but null results not reported.
• No a priori endpoint.
• Outcomes selected on basis of analysis of control group
data.
• These outcomes justified in terms of a literature in which
they have served as process variables for reduction of
distress that was not obtained in this trial.
*Giese-Davis et al., J Consult Clin Psych 2002

18. Misusing and Misunderstanding the
Profile of Mood States (POMS)
• Provides general mood score and several subscales.
• Allows multiple correlated tests of whether intervention
affects mood.
• Any positive findings emphasized and negative findings
and capitalization on chance ignored.
• Emphasized over CES-D when CES-D yields null
findings.
• CES-D preferable.
• Investigators tend to ignore normative data
suggesting that cancer patients entering trials have
mood as good or better than college students.

19. Rescuing a Null Trial With A Newly
Invented Outcome: Benefit Finding*
• A priori primary endpoint was distress.
• Primary analysis yielded null results, but subsequent
reports have reported a secondary analysis in which
there was an effect for a subgroup of patients.
• Subsequent reports give main emphasis to benefit
finding as an endpoint.
• Intervention not designed to affect benefit finding, no
theoretical reason for assuming an effect.
• Benefit finding has unknown clinical significance.
*Antoni et al, Health Psychology 2001

20. How Not to Solve the Problem of
Weak Effects
Dominant strategy involves bigger, more
intense and complex interventions
which:
• Aggravate problems of selection bias.
• Create problems of selective exposure
to interventions and breakdown in
adherence.
• Preclude elucidation of mechanism.

21. With possible exception of dying
patients, no obvious theoretical or
technical reason for assuming that
psychological interventions cannot
reduce distress---not even an
interesting question---if cancer
patients are selected for clinical need.

22. CONSORT
• Will help, but not a panacea.
• Provides an evaluation of
transparency, but not directly the
quality with which a trial has been
designed and executed.
• Biased analysis and interpretation
more obvious, but not eliminated.

23. What is Needed
• Registry of Trial Designs at NCI so that reader
can compare what is reported with what was
proposed.
• Reviewers start incorporating CONSORT NOW!
• Abstracts faithfully represent what was done in
trials NOW.
• Readers cultivate own critical appraisal skills.
• Graduate training provide more emphasis on
design, conduct, analysis, interpretation, and
reporting of clinical trials.

24. What is Needed
• Trials targeting patients with significant distress.
• Recruitment strategies based on accurate
estimates of prevalence of distress in
population.
• Comparisons of psychotherapeutic intervention
to access to information and improved
management of symptoms and side effects.
• Patient preference and randomized
encouragement designs.

25. What is Fair
• Given quality of evidence for efficacy of
intervention, don’t oversell.
• Not every cancer patient needs therapeutic
intervention, wrong to suggest that they do.
• Lower barrier to patients accessing intervention
if they want it, beware of aggressive outreach.
• Informed consent procedures should not
mislead patients concerning evidence for benefit
of treatments to which they are being recruited.

26. Simpler Interventions with Some
Hope of Dissemination
“Dissemination of efficacious interventions
into real-world settings may be hampered
by inclusion of many activities, techniques,
and strategies that go far beyond their
underlying theories” (Rotheram-Borus &
Duan, 2003).

27. Beware of Unnecessarily Complex
Interventions

28. Must Come to Terms with
Poor Quality of Existing
Literature

29. Take Down the Slums and Build Anew With
Studies With Adequate Designs, Suitable
Patient Populations, and Fair Reporting .