What is a systematic review

1. What is a Systematic Review?
In basic terms, a systematic review is a
protocol-driven, comprehensive literature
review, usually designed to answer a specific
clinical question.

2. Time Commitment?
 On average, systematic reviews require at
least 12 months of preparation.
Estimated timeline…
Month Activity -
1 – 2 Preparation of protocol.
3 – 8 Searches for published and unpublished studies.
2 – 3 Pilot test of eligibility criteria.
3 – 8 Inclusion assessments.
3 Pilot test of ‘Risk of bias’ assessment.
3 – 10 Validity assessments.
3 Pilot test of data collection.
3 – 10 Data collection.
3 – 10 Data entry.
5 – 11 Follow up of missing information.
8 – 10 Analysis.
1 – 11 Preparation of review report.
12 – Keeping the review up-to-date.

3. What is the difference…
between Literature Review and Systematic Review
 Systematic reviews generally answer very focused, PICO-based questions.
 Systematic reviews have a protocol in place prior to the literature review
beginning, including:
 The clinical question
 Specific inclusion and exclusion criteria
 Methods for assessing bias
 Methods for combining the data (e.g., via a meta-analysis)
 A systematic review literature search is pre-specified and designed to find all
relevant materials; difference is a literature review does not follow a pre-
specified protocol, nor does it need to be truly “comprehensive”.

4. What is the difference… cont.
between Literature Review and Systematic Review
 Systematic reviews publish, as part of their methods sections, the details of the
search strategy. Systematic review literature searches are ultimately replicable;
the search strategies (including database names and platforms, dates of the
search, all search terms, and any limits used) are published so that others who
want to redo the searches can find the same information.
 Regular literature review searches do not need to be replicable or even
reported, though on occasion, you may see a published search strategy or a list
of keywords the author used to search as part of a regular review’s
methodology. Just because a literature review does not publish a list of search
terms (which is not particularly helpful without the logic behind it) or a full
strategy does not make it a bad or “loosey-goosey” search–it just means the
authors did not need to make the search replicable for a systematic review.

5. What is… continued
 Systematic reviews are often the basis for a meta-analysis, where the data from
the materials fitting the pre-specified criteria are pooled and statistically
analyzed. Traditional literature reviews do not apply additional statistical
methods to the materials found.
 Systematic reviews take exponentially more time to do, from the search strategy
creation itself, to going through each retrieved citation in duplicate or triplicate,
to analyzing the data from the included articles.

6. Minimum Parameters…
1) At least two team members working on the review, ideally three team members.
 Team members needed in order to prevent bias.
 At least one member should ideally be a research librarian
2) Clearly defined question (PICO)
3) Pre-Specified Inclusion and Exclusion Criteria
 A systematic review is composed of only primary studies
4) Comprehensive Search of Literature with Clear Methods/Search Strategy
 Need to at least search
1) Databases of Published Literature
2) Grey Literature - degree dependent on question
3) Reference List of Key Articles
4) Cited Reference Search of Key Articles

7. Minimum Parameters…(Cont.)
5) Critical Evaluation of Included Articles/Studies
6) Quantitative or Qualitative Data Synthesis
7) Evidence Based Conclusions
Part of Criteria Come From:
Cook, D. J., Mulrow, C. D., & Haynes, R. B. (1997). Systematic reviews: synthesis of
best evidence for
clinical decisions. Annals of internal medicine,126(5), 376-380.

8. Before You Begin: Ask?
 Do I have a clearly defined clinical question with established inclusion and
exclusion criteria?
 Do I have a team of at least three people assembled?
 Do I have time to go through as many search results as we might find?
 Do I have resources to get foreign-language articles appropriately translated?
 Do I have the statistical resources to analyze and pool data?