Evaluates a meta analysis of family therapy interventions for families facing physical illness. The slide presentation and article is discussed in greater detail at http://jcoynester.wordpress.com/2013/08/12/interventions-for-the-family-in-chronic-illness-a-meta-analysis-i-like/

1. Anatomy of a
Meta-Analysis I Like
James C Coyne, PhD
University Medical Center, Groningen,
the Netherlands
jcoynester@gmail.com

2. This set of slides maps items from
the QUOROM (Quality of Reporting
of Meta-Analyses, Lancet) statement
into text from an article reporting a
meta analysis of family interventions
in chronic illness.

3. Mechthild Hartmann, Eva Bäznerk,
Beate Wild, Ivan Eisler, Wolfgang
Herzog. Effects of Interventions
Involving the Family in the Treatment
of Adult Patients with Chronic
Physical Diseases: A Meta-Analysis.
Psychotherapy and Psychosomatics,
2010, 79(3), 136-148.

4. Meta-analysis uses the same basic steps as all
research
 Formulating problem (are there enough
studies)
 Gathering studies (file drawer problem)
 Coding studies (creating data)
 Computing effect sizes (how to do for
nonstandard data)
 Analyzing data (lots of options)
 Interpreting and presenting results (graphics)

5. Like an experimental study, a report
of a meta-analysis should report
sufficient details of what was done so
that a reader can critically evaluate
what was done and how results were
interpreted.

6. Transparency and
Reproducibility

7. However, even more than in the case
of an experimental study, the meta-
analyses should report details of how
studies were coded and analyzed and
with what results so that readers can
re-analyze results and come to their
own independent conclusions.

8. The QUORUM statement (Quality
of Reporting of Meta-Analyses,
Lancet, 1999) is now PRISMA
(Preferred Reporting Items for
Systematic Reviews and Meta-
Analyses)

9. Quorom: Abstract
Objectives Uses a structured format states the clinical
question explicitly
Data Sources The databases and other information sources
Review Methods The selection criteria: methods for validity
assessment, data abstraction, study
characteristics, and quantitative data synthesis
Results Characteristics of the RCTs included and
excluded; qualitative and quantitative findings,
with sub-group analyses if appropriate
Conclusion The main results

10. Quorom: Introduction
The explicit clinical problem,
biological rationale for the intervention
and rationale for the review.

11. Quorom: Method
Searching The information sources in detail and any restrictions
(years considered, publication status, language of
publication) .
Selection The inclusion and exclusion criteria (population,
intervention, principal outcomes, study design) .
Validity assessment The criteria and process used.
Data abstraction The process or processes used (e.g. independently or in
duplicate) .
Study characteristics The type of study design, participants’ characteristics,
details of intervention, outcome definitions .
Quantitative data
synthesis
The principal measures of effect, method of
combining results, handling of missing data; how
statistical heterogeneity was assessed; a rationale for
any a priori sensitivity and sub-group analyses; and any
assessment of publication bias .

12. Quorom: Results
Trial flow Provides a meta-analysis profile
summarizing trial flow.
Study
characteristics
Presents descriptive data for each trial.
Quantitative data
synthesis
Reports agreement on the selection and
validity assessment; presents simple
summary results; presents data needed to
calculate effect sizes and confidence
intervals in intention-to-treat analyses.

13. Quorom: Discussion
Summarizes key findings; discusses clinical
inferences based on internal and external validity;
interprets the results in the light of the totality of
available evidence; describes potential biases in
the review process and suggests a future research
agenda.

14. Step 1: Problem Conceptualization and
Operationalization
The first step is to conceptualize the problem,
operationalize the variables, and create the
hypotheses. The problem statement should
include a specification of the relevant research
literature and the major independent and
dependent variables (Lipsey & Wilson, 2001).

15. MH: A separate analysis of chronic physical diseases
is reasonable, as evidence suggests that this patient
group is more resistant to psychosocial interventions
than those presenting mental diseases [21, 22] . In
view of the limited recovery rates for traditional
biomedical approaches, there is a broad consensus
that additional non-pharmacological interventions
will be crucial in improving the health outcomes in
chronic physical diseases [2, 23].
Here, we perform a systematic review and meta-
analysis of randomised controlled trials (RCTs) to
assess the effects of family-oriented interventions
designed to improve the health of adults with chronic
physical disease.

16. Step 2: Data Collection and Processing.
Given that numerous articles will likely be
identified, procedures should be established
for tracking article collection and organizing
citation information (e.g., a researcher may
choose to create a bibliographic database).

17. Study Selection and Search Strategy
MH: A study protocol and inclusion/exclusion
criteria were established before the search. All
psychosocial interventions with family involvement
provided by health professionals to improve physical
health outcomes were included when classifiable as
(psycho)education or addressing relationships. Trials
based on information brochures or films without
additional education or discussion provided by a
health professional were excluded. Trials focused on
comparing hospital versus home-based care were
also excluded.

18. MH: We hypothesized that characteristics of
the intervention and the samples may explain
the heterogeneity in the effect size. We
therefore conducted subgroup analyses of the
disease groups, types of interventions
(psychoeducation, addressing relationships)
and types of family members (spouse vs.
mixed family members) to identify possible
moderators. The influence of the intervention
intensity was also assessed via meta-
regression analysis.

19. Validity
 Have studies been sought thoroughly:
 Medline and other relevant bibliographic database
 Cochrane controlled clinical trials register
 Foreign language literature
 "Grey literature" (unpublished or un-indexed reports:
theses, conference proceedings, internal reports, non-
indexed journals, pharmaceutical industry files)
 Reference chaining from any articles found
 Personal approaches to experts in the field to find
unpublished reports
 Hand searches of the relevant specialized journals.

20. MH: Our highly sensitive search strategy combined free-text
words and medical subject heading (MeSH) terms targeting
‘chronic diseases’, ‘family’, ‘information / education /
intervention / psychotherapy’ and ‘randomised controlled
trials’. The search was conducted with the following electronic
databases: the Cochrane Central Register of Controlled Trials,
Cochrane Database of Systematic Reviews (issue II, 2007),
MEDLINE (1950), PsycIndex (1977), PsycINFO (1887) and
CINAHL (1982). Finally, leading scientists in the field and the
study authors of registered trials were contacted in order to
locate trials not registered in the databases, including
unpublished trials. In addition, reference lists of previously
published meta-analyses or systematic reviews of all included
studies were searched.

21. MH: Data Extraction and Quality Assessment.
To determine which trials to assess, 2 review
authors independently scanned the titles and
abstracts of every retrieved record. Every
article identified as relevant by at least 1 of the
review authors was retrieved in full for
relevance screening, which was also
independently conducted by 2 individuals.
Disagreements about eligibility were resolved
by discussion with the other authors.

22. MH: Study characteristics were extracted by 2
review authors using a standardised form
developed before the search. If the details of
the data were unclear, the study authors were
contacted for clarification. We contacted the
authors of 18 papers and received replies from
12, of whom 6 were able to provide the
statistical data needed. We assessed the
methodological quality of the included trials
with regard to randomization procedure,
allocation concealment, loss to follow-up and
intention-to-treat analysis [26].

23. Step 3: Setting Inclusion and Exclusion
Criteria
Planning for the inclusion or exclusion criteria is perhaps the
most important component of meta-analysis. Such criteria are
directly related to internal/external validity and
generalizability (Gliner et al., 2003). Factors such as sampling
methods, research methods, time frames, publication types,
and cultural/language differences of studies should all be
considered (Cooper & Hedges, 1994).
These decisions are heavily dependent on a researcher’s
purposes, goals, and available resources. Lipsey and Wilson
(2001) suggest that researchers prepare a detailed, written
specification of the criteria a study must meet for inclusion in
the meta-analysis.

24. Validity
 Have inclusion and exclusion criteria for
studies been stated explicitly, taking
account of the patients in the studies, the
interventions used, the outcomes recorded
and the methodology?

25. Validity
 Have the authors considered the
homogeneity of the studies: the idea that the
studies are sufficiently similar in their
design, interventions and subjects to merit
combination.
• this is done either by eyeballing graphs
like the forest plot or by applications of
chi-square tests (Q test)

26. MH: Given the great variety of diseases,
interventions and outcomes, we decided a
priori to use random-effects models to
calculate the across-study effect size. To
assess heterogeneity, we used the I2 statistic
[26]. Publication bias was assessed through
visual inspection of the funnel plot.

27. Step 4: Analysis and Results
The basic analytic goals of meta-analysis are
to: (a) combine and analyze the distribution of
effect sizes and (b) examine the relationship
between effect sizes and other descriptive
variables to understand the variability of effect
sizes across studies.

28. Step 4: Analysis and Results
The four basic steps are:
1. Create independent effect sizes for each
study.
2. Compute the weighted mean of effect sizes
using inverse variance weights.
3. Determine the confidence interval for the
mean.
4. Analyze for homogeneity.

29. MH: The quality of the included RCTs differed
considerably. Intention-to-treat analyses were
conducted in 29 of the 52 studies. Ten failed to
conduct intention-to-treat analyses, and 13 made no
statements in this respect. Concealment of allocation
could only be verified in 18 studies, primarily because
the majority of study authors did not provide this
information. Similarly, descriptions of the
randomization procedure were reported in only 20
studies. Most studies (n = 45) disclosed a loss to
followup, and a high dropout rate of over 30% was
found in only 2 trials.
There was no evidence for publication bias upon
visual inspection of the funnel plots for all outcomes.

30. MH: Sensitivity analyses were undertaken to
determine whether the pooled effect sizes are
dependent on study quality or on our decision
to use specific outcome hierarchies. All
analyses were therefore redone for alternative
hierarchies and each hierarchy level
separately. We also recalculated the effect
sizes using only trials fulfilling the highest
methodological standards.

31. MH: The results of our meta-analysis suggest that family-
oriented interventions in chronic physical diseases are more
effective for improving physical health outcomes and reducing
mental health problems in both patients and caregivers than
commonly used treatments. The pooled overall estimate was
0.32 (Hedges’ g ) for physical health of the patient and 0.28
(Hedges’ g ) for patient mental health. The interventions can
further be considered useful to promote better health among
family members (Hedges’ g = 0.35). All results can be
described as small [30] statistically significant effects
favouring family-oriented interventions. In light of the large
variety in format, content and the intensity of the intervention,
the effect was extremely stable. It corresponds to an OR of
1.72–1.84, meaning that patients in the family involvement
group had a 72–84% higher chance of improved health
compared to the usual care group.

32. Step 5: Discussion
Summarizes key findings; discusses clinical
inferences based on internal and external
validity; interprets the results in the light of
the totality of available evidence; describes
potential biases in the review process and
suggests a future research agenda.

33. MH: The primary limitation of our review is common
to all meta-analyses, that is the dependency on the
quality of the primary studies. We found that the
methods for conducting these trials were suboptimal.
By restricting our analysis to RCTs, however, we
used the best evidence in this field for our review. To
account for methodological weaknesses, we
conducted sensitivity analyses that demonstrated
that study quality did not affect the effect size
estimates in a statistically significant manner. A
second limitation may be the high degree of
heterogeneity, which was present in the primary
studies and was reflected in the I2 statistic.

34. MH: Our current results may have greater
implications for future research in chronic
medical care than for individual clinical
decisions.